-- Logs begin at Thu 2022-07-21 21:33:25 UTC, end at Thu 2022-07-21 23:26:04 UTC. -- Jul 21 22:18:05.083578 dv-cinder-850696-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 22:18:05 2022] *** Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: nodename: dv-cinder-850696-1-iscsi Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: machine: x86_64 Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: clock source: unix Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: pcre jit disabled Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: detected number of CPU cores: 4 Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: current working directory: / Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: detected binary path: /usr/bin/uwsgi-core Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: your processes number limit is 47680 Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: your memory page size is 4096 bytes Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: detected max file descriptor number: 2048 Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: lock engine: pthread robust mutexes Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: thunder lock: enabled Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 22:18:05.095573 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 22:18:05.137967 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: Python main interpreter initialized at 0x55971abec9c0 Jul 21 22:18:05.137967 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: python threads support enabled Jul 21 22:18:05.137967 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: your server socket listen backlog is limited to 100 connections Jul 21 22:18:05.137967 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: your mercy for graceful operations on workers is 90 seconds Jul 21 22:18:05.138279 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 22:18:05.138386 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: *** Operational MODE: preforking *** Jul 21 22:18:05.138659 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: *** uWSGI is running in multiple interpreter mode *** Jul 21 22:18:05.138659 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: spawned uWSGI master process (pid: 80307) Jul 21 22:18:05.138855 dv-cinder-850696-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 22:18:05.139407 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: spawned uWSGI worker 1 (pid: 80315, cores: 1) Jul 21 22:18:05.139733 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: spawned uWSGI worker 2 (pid: 80316, cores: 1) Jul 21 22:18:05.139884 dv-cinder-850696-1-iscsi devstack@keystone.service[80307]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 22:18:10.702392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 22:18:10.702392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: from cryptography.utils import int_from_bytes Jul 21 22:18:10.703158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 22:18:10.703158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: from cryptography.utils import int_from_bytes Jul 21 22:18:10.753395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 22:18:10.753605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 22:18:10.753790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] command line args: [] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 22:18:10.754055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 22:18:10.754236 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ================================================================================ {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 22:18:10.760038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] admin_token = **** {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.760038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] config_dir = [] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.760038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.760038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] config_source = [] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.760038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] debug = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.760038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.760038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.761959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.761959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.761959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.761959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] list_limit = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.761959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] log_config_append = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.761959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.761959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] log_dir = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.761959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] log_file = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.761959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.762846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.762846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.762846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.762846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.762846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.762846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.762846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.763426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.763426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.763426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.763426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.763426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.763426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.763426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.763426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.763426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] publish_errors = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.764598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.764598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.764598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.764598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.764598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.764598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] standard_threads = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.764598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.764598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.764598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.765600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] use_journal = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.765600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] use_json = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.765600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] use_stderr = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.765600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] use_syslog = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.765600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.765600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.766382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.766382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.766382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.766758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.766758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.766977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.767167 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.767421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.767669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.767924 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.768162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.768413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.768691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.768942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.769194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.769435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.769694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.770117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.770960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.771267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.771723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.772000 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.772261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.772539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.772780 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.773017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.773253 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.773494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.773776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.773999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.774233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.774470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.774975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.775227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.775469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.775724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.775962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.776215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.776459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.776716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.776953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.777191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.777427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.777671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.777924 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.778161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.778402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.778638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.778885 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.779126 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.779383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.779620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.779863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.780102 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.780339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.780600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.780843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.781080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.781337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.781578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.781815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.782052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.782319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.782563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.782802 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.783282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.783560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.783804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.783965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 22:18:10.783965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: from cryptography.utils import int_from_bytes Jul 21 22:18:10.784589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.784589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.784811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 22:18:10.784811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: from cryptography.utils import int_from_bytes Jul 21 22:18:10.785016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.785016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.785016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.785357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.785468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.785785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.786226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.786470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.786714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.786953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.787196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.787433 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.787703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.787950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.788208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.788451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.788726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.788975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.789222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.789464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.789699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.789936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.790171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.790409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.790645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.790890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.791129 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.791382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.791620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.791881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.792145 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.792392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.792650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] auth.external = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.792885 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.793186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.793422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.793657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] auth.password = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.793893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] auth.token = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.794137 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.794376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.794626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.794861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.795112 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.795362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.795597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.795835 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] credential.caching = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.796086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.796340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.796616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.796858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.797108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.797349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.797619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.797877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.798118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.798376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.798672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.800588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.800893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.801155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.801449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.801830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] federation.caching = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.802085 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.802209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.802425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.802663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.802943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.803166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.803435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.803681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.803943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.804202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.804606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.804748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.caching = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.804941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.806534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.806534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.806534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.806534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.806534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.806534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.807442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.807442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.807442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.807442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.808114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.808114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.808431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.808431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.808821 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.809029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.809216 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.809398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.809595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.809841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.810082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.810323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.810583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.810813 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.811062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.811313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.811552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.811804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.812055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.812293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.812640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.812891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.813134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.813376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.813611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.813852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.814093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.814344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.814584 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.814826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.815079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.815336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.815570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.815806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.816056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.816308 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.816558 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.817312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.817564 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.817798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.818045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.818311 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.818543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.818801 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.819048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.819297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.819536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.819768 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.820005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.820244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.820561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.820753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.820976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.821235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.821485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.821735 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.821984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.822226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.822478 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.822718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.822957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.823197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.823429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.823684 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.823928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.824168 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.824426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.824689 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.824939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.825178 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.825418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.825663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.825914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.826157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.826393 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.826633 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.826877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] resource.caching = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.827129 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.827380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.827617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.827868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.828121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.828360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.828636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.828878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.829120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.829359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] role.caching = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.829609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] role.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.829842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.830093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.830342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.830593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.830841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.831090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.831338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.831586 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.831836 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.832070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.832319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.833140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.833390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.833642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.834078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.834314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.834436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 22:18:10.834554 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.834670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 22:18:10.834843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] command line args: [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 22:18:10.834963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.835080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.835080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.835707 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 22:18:10.836001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.836254 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ================================================================================ {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 22:18:10.837643 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.837772 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] admin_token = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.837892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.838024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] config_dir = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.838144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.838260 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.838375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.838490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] config_source = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.838490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] debug = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.838719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.838837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.838983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.839101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.839226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.839344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.839459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.839574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.839689 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.839804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.839928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.840045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.840161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.840277 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_config_append = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.840393 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.840524 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.840524 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_dir = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.840760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] token.caching = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.840876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_file = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.840991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.841114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.841231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.841355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.841471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.841587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.841702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.841817 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.841935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.842059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.842176 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.842292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.842409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.842532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.842654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.842777 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.842894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.843009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.843125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.843240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.843365 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.843490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.843607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.843722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.843838 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.843937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.844052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.844179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.844295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.844417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.844585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.844585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.844815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.844815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] publish_errors = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.845035 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.connection = **** {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.845160 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.845365 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.845485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.845609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.845726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.845841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.845841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.846057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.846173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.846299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.846416 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.846531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] standard_threads = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.846646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.846761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.846875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.846999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.847116 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.847232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.847347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.847462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] use_journal = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.847462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] use_json = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.847687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.847804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] use_stderr = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.847919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.848034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] use_syslog = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.848185 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.848305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:18:10.848430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.848567 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.848674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.848788 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.848920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 22:18:10.849773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.849773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.850731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.850731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.850731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.850731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.851160 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.851160 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.851392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.851625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.851894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.852153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.852510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.852706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.853068 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.853187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.853360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.853613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.854631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.854631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.854631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.854631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.855410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.855410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.855410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.855742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.855742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.855960 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.856136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.856371 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.856634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.856876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.857119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.857377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.857611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.857846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.858080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.858319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.858559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.858810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.859044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.859285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.859524 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.859762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.860005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.860245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.860501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.860769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.861003 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.861245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.861481 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.861748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.861990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.862244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.862469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.862713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.862984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.863228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.863482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.863792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.864713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.864971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.865217 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.865472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.865727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.865956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.866207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.866457 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.866690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.866935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.867178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.867415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.867684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.867928 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.868174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.868416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.868693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.868937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.869176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.869412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.869645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.869877 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.870112 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.870365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.870599 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.870842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.871077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.871328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.871561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.871824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.872090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.872330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.872614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.external = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.872804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.873110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.873366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.873557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.password = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.873791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.token = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.874037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.874276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.874528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.874764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.875017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.875265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.875497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.875739 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.875993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.876242 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.876528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.876752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.877002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.877242 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.877521 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.877778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.878016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.878272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.878560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.878804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.879070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.879863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.880229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:10.880527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:10.880916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.881119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:10.881339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.881339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.881708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.881708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.882073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.882263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.882463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.882650 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.882832 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.883077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.883316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.883566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.883865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.884040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.884290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.884555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.884798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.885049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.885263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.885508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.885889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.886146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.886381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.886616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.886852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.887103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.887357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.887609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.887876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.888127 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.888400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.888667 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.888912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.889148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.889389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.889594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:10.889783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.889971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.890155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.890343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.890534 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.890775 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.891026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.891699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.891699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.892066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.892066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.892436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.892436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.892837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.893017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.893199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.893382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.893588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.893834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.894085 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.894336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.894570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.894805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.895865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.896613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.896869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.897119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.897365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.897607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.897862 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.898132 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.898381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.898630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.898884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.899133 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.899373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.899855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.900100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.900338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.900602 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.900836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.901084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.901332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.901580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.901829 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.902073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.902305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.902554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.902792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.903028 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.903264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.903498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.903744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.903990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.904227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.904541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.904727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.904980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.905216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.905451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.905689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.905936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.906178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.906409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.906648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.906886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.907138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.907388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.907622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.907872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.908128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.908377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.908653 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.908893 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.909137 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.909374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] role.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.909624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] role.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.909857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.910110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.910359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.910763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.911600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.911880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.912157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.912412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.912696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.912936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.913190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.913440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.913693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.913942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.914374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.914608 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.914860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.915108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.915361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.915611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.915847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.916106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.916358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.916624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.916861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.917095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.917328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.917580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.917838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.918088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.918326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.918570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.918802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.919025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:10.919225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.919428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.919616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:10.919809 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.919809 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.920172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.920172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.920574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.920759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.920948 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.921134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.921371 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.921611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.921846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.922095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.922344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.922583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.922834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.923130 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.923392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.connection = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.923635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.923880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.924122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.924363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.924607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.924852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.925069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.925313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.925557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.925793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.926034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.927424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.927714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.927975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.928235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.928506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.928756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.929003 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:18:10.929312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 22:18:10.957433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:10.958202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:10.959622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:10.968511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:10.991333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:10.991543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.064442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.119104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.119382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.127604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.167940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.168200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.188467 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.188717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.188932 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.189152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.189372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.189588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.212840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.213069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.213212 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.213413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.213636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.213854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.216004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.216241 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.216470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.216724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.216945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.217167 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.217383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.229613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.229872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.230106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.230332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.230560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.230782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.231004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.231222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.239847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.242664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.242897 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.243096 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.243227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.243348 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.243469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.243598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.243723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.243723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.243946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.244063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.244179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.244179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.244410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:18:11.261528 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.261791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 22:18:11.262071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.262300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.262473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 22:18:11.262622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 22:18:11.262814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.263131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.263335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.263533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 22:18:11.263720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.263906 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.264090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.264272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.264272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.264272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.275313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.275596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.275869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.276155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.276320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.276630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.276903 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.277416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.284402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.287049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.288278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.288588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.288875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.289282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.289570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.289833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.290105 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.290381 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.290502 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.290655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.290788 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.290930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.291055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.291200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.291332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.291464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.291725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.291995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.292267 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.292563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.292836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.293214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.318079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.319882 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.320169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.321921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.323731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.324017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.325856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.326151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.326440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.329859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.330164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.330455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.334661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.334976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.339280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.339585 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.339852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.340165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.340699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.340901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.341389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.341389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.341754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.341943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.344636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.344951 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.345239 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.345540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.345826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.346102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.346505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.346792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.361613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.361893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.366992 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.368213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.368541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.370917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.371226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.373472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.376436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.377336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.377646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.380270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.383413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.384308 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.384659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.384971 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.385271 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.385560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.385843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.387218 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.391041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.391356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.391649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.392069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.392352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.392662 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.406601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.407013 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.407288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.407556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.415398 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.415398 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.415746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.415847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.416502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.418293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.418570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.421728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.422016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.422299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.422583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.422896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.424181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.425981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.426408 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.431769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.431769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.431769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.431769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.431769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.436662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.436969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.437254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.437539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.437825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.438120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.438411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.446155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.446455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.446746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.447029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.447312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.447605 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.447884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.459389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.462917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.466303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.467515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.467954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.469731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.470025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.470309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.472949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.477275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.478262 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.478922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.479960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.480611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.480901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.481195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.483369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.483687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.483984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.484276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.489645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.491516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.494272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.494579 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.494874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.495011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.495220 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.495508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.495790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.503193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.503555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.503754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.504017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.504862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.505335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.505692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.505970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.506259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.506533 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.506725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.506919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.507110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.507295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.507498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.507695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.507695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.515819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:18:11.516265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.516461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.516918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.517199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.517592 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.517797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.518067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.518346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.518621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.518897 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.519159 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.519425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.519702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:18:11.548001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55971abec9c0 pid: 80316 (default app) Jul 21 22:18:11.552373 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c2ac-d14d-4829-88a3-b255b015b31c None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:11.552961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c2ac-d14d-4829-88a3-b255b015b31c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:11.552961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c2ac-d14d-4829-88a3-b255b015b31c None None] PATH_INFO: `/v3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:11.558161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Thu Jul 21 22:18:11 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:11.562497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55971abec9c0 pid: 80315 (default app) Jul 21 22:18:19.854402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48a11d-7aee-4e3d-91d4-4d6bf5fe8794 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:19.855019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48a11d-7aee-4e3d-91d4-4d6bf5fe8794 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:19.855019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48a11d-7aee-4e3d-91d4-4d6bf5fe8794 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:19.856308 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 2/2] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:19.865668 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ccb03-835f-4941-a025-12a4bdcdda6f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:19.866217 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ccb03-835f-4941-a025-12a4bdcdda6f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:19.866340 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ccb03-835f-4941-a025-12a4bdcdda6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:19.940964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: INFO dbcounter [None req-de2ccb03-835f-4941-a025-12a4bdcdda6f None None] Registered counter for database keystone Jul 21 22:18:19.965049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG oslo_db.sqlalchemy.engines [None req-de2ccb03-835f-4941-a025-12a4bdcdda6f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80315) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 22:18:19.972046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writer thread running {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 22:18:20.175190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG passlib.handlers.bcrypt [None req-de2ccb03-835f-4941-a025-12a4bdcdda6f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80315) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 22:18:20.175613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG passlib.handlers.bcrypt [None req-de2ccb03-835f-4941-a025-12a4bdcdda6f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80315) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 22:18:20.233760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-de2ccb03-835f-4941-a025-12a4bdcdda6f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:20.235430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-de2ccb03-835f-4941-a025-12a4bdcdda6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:20.367082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1/3] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 506 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 22:18:20.455918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48a11d-7aee-4e3d-91d4-4d6bf5fe8794 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:20.528290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: INFO dbcounter [None req-2b48a11d-7aee-4e3d-91d4-4d6bf5fe8794 None None] Registered counter for database keystone Jul 21 22:18:20.549848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG oslo_db.sqlalchemy.engines [None req-2b48a11d-7aee-4e3d-91d4-4d6bf5fe8794 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80316) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 22:18:20.557280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writer thread running {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 22:18:20.679575 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48a11d-7aee-4e3d-91d4-4d6bf5fe8794 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:20.684134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e887652-4cf8-445d-98d0-1860a38edc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:20.684779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e887652-4cf8-445d-98d0-1860a38edc7d admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:20.685067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e887652-4cf8-445d-98d0-1860a38edc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:20.685341 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e887652-4cf8-445d-98d0-1860a38edc7d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:20.695061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e887652-4cf8-445d-98d0-1860a38edc7d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:20.790579 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6e887652-4cf8-445d-98d0-1860a38edc7d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 22:18:20.790919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e887652-4cf8-445d-98d0-1860a38edc7d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:20.802870 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 3/4] 10.222.0.42 () {58 vars in 1324 bytes} [Thu Jul 21 22:18:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:18:20.809197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ccb03-835f-4941-a025-12a4bdcdda6f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:20.827492 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ccb03-835f-4941-a025-12a4bdcdda6f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:20.832781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e29f3-d9ca-4c6a-aa23-d7d87f8f854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:20.834155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e29f3-d9ca-4c6a-aa23-d7d87f8f854a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:20.834493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e29f3-d9ca-4c6a-aa23-d7d87f8f854a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:20.834821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e29f3-d9ca-4c6a-aa23-d7d87f8f854a admin admin] PATH_INFO: `/v3/projects/5705979642f541aa9229c56505a6ee6e` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:20.836445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e29f3-d9ca-4c6a-aa23-d7d87f8f854a admin admin] RBAC: Authorizing `identity:get_project(project_id=5705979642f541aa9229c56505a6ee6e)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:20.889098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7e4e29f3-d9ca-4c6a-aa23-d7d87f8f854a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 22:18:20.889919 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e29f3-d9ca-4c6a-aa23-d7d87f8f854a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:20.895222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 2/5] 10.222.0.42 () {56 vars in 1207 bytes} [Thu Jul 21 22:18:20 2022] GET /identity/v3/projects/5705979642f541aa9229c56505a6ee6e => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:23.171075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e290bc7-98df-43ba-ad7d-4d4f14b79643 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.171640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e290bc7-98df-43ba-ad7d-4d4f14b79643 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.171640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e290bc7-98df-43ba-ad7d-4d4f14b79643 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.176188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 4/6] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:23.185846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece3ebd-14e0-4411-9be6-697e7f10490a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.186380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece3ebd-14e0-4411-9be6-697e7f10490a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.186784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece3ebd-14e0-4411-9be6-697e7f10490a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.218640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-dece3ebd-14e0-4411-9be6-697e7f10490a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:23.221450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dece3ebd-14e0-4411-9be6-697e7f10490a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:23.235766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 3/7] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 22:18:23.276756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e290bc7-98df-43ba-ad7d-4d4f14b79643 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:23.290455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e290bc7-98df-43ba-ad7d-4d4f14b79643 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:23.295435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80db8a0-d7e3-41e7-afaa-ca9b98f57884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:23.296117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80db8a0-d7e3-41e7-afaa-ca9b98f57884 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.296403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80db8a0-d7e3-41e7-afaa-ca9b98f57884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.296726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80db8a0-d7e3-41e7-afaa-ca9b98f57884 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.306886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80db8a0-d7e3-41e7-afaa-ca9b98f57884 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:23.308823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80db8a0-d7e3-41e7-afaa-ca9b98f57884 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:23.320772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 5/8] 10.222.0.42 () {58 vars in 1324 bytes} [Thu Jul 21 22:18:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:18:23.327142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece3ebd-14e0-4411-9be6-697e7f10490a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:23.337116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece3ebd-14e0-4411-9be6-697e7f10490a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:23.341815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a102cfe-2d1b-4514-ad70-191ed7621caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:23.342719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a102cfe-2d1b-4514-ad70-191ed7621caa admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:23.343009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a102cfe-2d1b-4514-ad70-191ed7621caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:23.343293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a102cfe-2d1b-4514-ad70-191ed7621caa admin admin] PATH_INFO: `/v3/users/e2841614037940fea030f1d6c1eca499` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:23.344848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a102cfe-2d1b-4514-ad70-191ed7621caa admin admin] RBAC: Authorizing `identity:get_user(user_id=e2841614037940fea030f1d6c1eca499)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:23.346797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a102cfe-2d1b-4514-ad70-191ed7621caa admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:23.348609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 4/9] 10.222.0.42 () {56 vars in 1198 bytes} [Thu Jul 21 22:18:23 2022] GET /identity/v3/users/e2841614037940fea030f1d6c1eca499 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:25.639451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066805b2-3ab0-4815-a5bf-e33930e9d201 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.640117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066805b2-3ab0-4815-a5bf-e33930e9d201 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.640117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066805b2-3ab0-4815-a5bf-e33930e9d201 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.641424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 6/10] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:25.648605 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38215e5-2205-415c-847e-cc99bed0c887 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.649664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38215e5-2205-415c-847e-cc99bed0c887 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.649664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38215e5-2205-415c-847e-cc99bed0c887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.668794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f38215e5-2205-415c-847e-cc99bed0c887 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:25.670029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f38215e5-2205-415c-847e-cc99bed0c887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:25.689240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a3cad-ece3-4d33-bb18-022a643b96f3 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.689621 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a3cad-ece3-4d33-bb18-022a643b96f3 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.690140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a3cad-ece3-4d33-bb18-022a643b96f3 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.693346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 7/11] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:25.701211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f79b5e-4827-44ec-b9a7-d4726f8d5bb6 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.701716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f79b5e-4827-44ec-b9a7-d4726f8d5bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.702222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f79b5e-4827-44ec-b9a7-d4726f8d5bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.766075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 5/12] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:25.776789 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG passlib.handlers.bcrypt [None req-61f79b5e-4827-44ec-b9a7-d4726f8d5bb6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80316) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 22:18:25.777527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG passlib.handlers.bcrypt [None req-61f79b5e-4827-44ec-b9a7-d4726f8d5bb6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80316) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 22:18:25.791800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38215e5-2205-415c-847e-cc99bed0c887 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:25.818562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:25.832630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-61f79b5e-4827-44ec-b9a7-d4726f8d5bb6 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:25.835551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-61f79b5e-4827-44ec-b9a7-d4726f8d5bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:25.923673 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38215e5-2205-415c-847e-cc99bed0c887 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:25.968641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 8/13] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:25.978433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc9e09-6730-4f92-9782-a35f6a29ec69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:25.979360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc9e09-6730-4f92-9782-a35f6a29ec69 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:25.979752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc9e09-6730-4f92-9782-a35f6a29ec69 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:25.980123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc9e09-6730-4f92-9782-a35f6a29ec69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:25.992991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f79b5e-4827-44ec-b9a7-d4726f8d5bb6 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:25.994743 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc9e09-6730-4f92-9782-a35f6a29ec69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-cecc9e09-6730-4f92-9782-a35f6a29ec69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:26.006926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:26.008911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:18:26.009923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 6/14] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:18:26.020350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc9e09-6730-4f92-9782-a35f6a29ec69 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:26.110936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:26.132074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc9e09-6730-4f92-9782-a35f6a29ec69 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.154596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f79b5e-4827-44ec-b9a7-d4726f8d5bb6 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.158976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:26.169795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c44d2-f8c3-4937-b2c4-c4f839e73511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.170529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c44d2-f8c3-4937-b2c4-c4f839e73511 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.170831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c44d2-f8c3-4937-b2c4-c4f839e73511 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.171055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c44d2-f8c3-4937-b2c4-c4f839e73511 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.171838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c44d2-f8c3-4937-b2c4-c4f839e73511 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:26.172155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c44d2-f8c3-4937-b2c4-c4f839e73511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:26.173657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c44d2-f8c3-4937-b2c4-c4f839e73511 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:26.188380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 7/15] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:26.194496 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c44d2-f8c3-4937-b2c4-c4f839e73511 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:26.217564 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883caa6-4d1b-4e57-a129-1ae779565e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.219269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883caa6-4d1b-4e57-a129-1ae779565e69 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.219736 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883caa6-4d1b-4e57-a129-1ae779565e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.220265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883caa6-4d1b-4e57-a129-1ae779565e69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.315518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883caa6-4d1b-4e57-a129-1ae779565e69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:26.317712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883caa6-4d1b-4e57-a129-1ae779565e69 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:26.318670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c44d2-f8c3-4937-b2c4-c4f839e73511 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.361278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06df458-a816-480f-a486-cb74ddc976e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.362350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06df458-a816-480f-a486-cb74ddc976e1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.362684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06df458-a816-480f-a486-cb74ddc976e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.363546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06df458-a816-480f-a486-cb74ddc976e1 None admin] PATH_INFO: `/v3/users/e2841614037940fea030f1d6c1eca499` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.365299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06df458-a816-480f-a486-cb74ddc976e1 None admin] RBAC: Authorizing `identity:get_user(user_id=e2841614037940fea030f1d6c1eca499)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:26.367038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06df458-a816-480f-a486-cb74ddc976e1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:26.369454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 8/16] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:26 2022] GET /identity/v3/users/e2841614037940fea030f1d6c1eca499 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:26.375850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06df458-a816-480f-a486-cb74ddc976e1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:26.396312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:26.451574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 9/17] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:18:26.458488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883caa6-4d1b-4e57-a129-1ae779565e69 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:26.462269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:26.514563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06df458-a816-480f-a486-cb74ddc976e1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.563393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924548e9-e316-4cca-870f-e0b73499ea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.564818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924548e9-e316-4cca-870f-e0b73499ea32 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.565315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924548e9-e316-4cca-870f-e0b73499ea32 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.566711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924548e9-e316-4cca-870f-e0b73499ea32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.568450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924548e9-e316-4cca-870f-e0b73499ea32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:26.571213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924548e9-e316-4cca-870f-e0b73499ea32 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:26.574202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 9/18] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:26.578197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883caa6-4d1b-4e57-a129-1ae779565e69 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.580818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924548e9-e316-4cca-870f-e0b73499ea32 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:26.617636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673c0ca-d776-4f96-8b31-74d2034e7a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.618505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673c0ca-d776-4f96-8b31-74d2034e7a87 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.618797 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673c0ca-d776-4f96-8b31-74d2034e7a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.619079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673c0ca-d776-4f96-8b31-74d2034e7a87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.642632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673c0ca-d776-4f96-8b31-74d2034e7a87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:26.644878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673c0ca-d776-4f96-8b31-74d2034e7a87 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-d673c0ca-d776-4f96-8b31-74d2034e7a87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:18:26.667982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:26.669414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:26.670759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:18:26.670759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:18:26.670759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:18:26.670759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:18:26.670759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 10/19] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:18:26.675432 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673c0ca-d776-4f96-8b31-74d2034e7a87 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:26.686894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:26.717095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924548e9-e316-4cca-870f-e0b73499ea32 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.749061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:26.762686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf60f65-02f2-45e6-b2e8-2a7281971083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.763489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf60f65-02f2-45e6-b2e8-2a7281971083 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.763867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf60f65-02f2-45e6-b2e8-2a7281971083 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.764248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf60f65-02f2-45e6-b2e8-2a7281971083 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.765429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf60f65-02f2-45e6-b2e8-2a7281971083 None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.domain.id=default, user.id=e2841614037940fea030f1d6c1eca499) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:26.765804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf60f65-02f2-45e6-b2e8-2a7281971083 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:26.767439 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf60f65-02f2-45e6-b2e8-2a7281971083 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:26.778029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 10/20] 10.222.0.42 () {56 vars in 1322 bytes} [Thu Jul 21 22:18:26 2022] GET /identity/v3/role_assignments?user.id=e2841614037940fea030f1d6c1eca499&scope.domain.id=default&role.id=f957e09d047b47cfa0891d25b4181263 => generated 233 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:26.810472 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673c0ca-d776-4f96-8b31-74d2034e7a87 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:26.864765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d14616-5247-4dde-91cb-e0e18022ecdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:26.865557 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d14616-5247-4dde-91cb-e0e18022ecdd None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:26.865907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d14616-5247-4dde-91cb-e0e18022ecdd None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:26.866279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d14616-5247-4dde-91cb-e0e18022ecdd None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:26.867179 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d14616-5247-4dde-91cb-e0e18022ecdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:26.867563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d14616-5247-4dde-91cb-e0e18022ecdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:26.869225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d14616-5247-4dde-91cb-e0e18022ecdd None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:26.891799 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 11/21] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:30.046989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008c71d-c79c-4c3e-be99-3d463ddaba86 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:30.048000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008c71d-c79c-4c3e-be99-3d463ddaba86 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:30.048518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008c71d-c79c-4c3e-be99-3d463ddaba86 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:30.050949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 11/22] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:30.057001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc90a68-00d3-417c-ab40-598b79906e43 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:30.057926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc90a68-00d3-417c-ab40-598b79906e43 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:30.058407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc90a68-00d3-417c-ab40-598b79906e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:30.083233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-dbc90a68-00d3-417c-ab40-598b79906e43 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:30.084874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-dbc90a68-00d3-417c-ab40-598b79906e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:30.182962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 12/23] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:30.218894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008c71d-c79c-4c3e-be99-3d463ddaba86 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:30.396407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008c71d-c79c-4c3e-be99-3d463ddaba86 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:30.444518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7d089-dc6c-48d1-a3a0-7c6f80958719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:30.445230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7d089-dc6c-48d1-a3a0-7c6f80958719 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:30.445530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7d089-dc6c-48d1-a3a0-7c6f80958719 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:30.445813 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7d089-dc6c-48d1-a3a0-7c6f80958719 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:30.446611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7d089-dc6c-48d1-a3a0-7c6f80958719 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:30.448063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7d089-dc6c-48d1-a3a0-7c6f80958719 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:30.555890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15118af-14b0-4284-80c0-3dfda2bd88cd None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:30.556592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15118af-14b0-4284-80c0-3dfda2bd88cd None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:30.557075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15118af-14b0-4284-80c0-3dfda2bd88cd None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:30.558969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:30.567021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21e317-afee-4f94-a982-acf1ddff3a73 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:30.567627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21e317-afee-4f94-a982-acf1ddff3a73 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:30.568134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21e317-afee-4f94-a982-acf1ddff3a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:30.605335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ae21e317-afee-4f94-a982-acf1ddff3a73 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:30.606960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ae21e317-afee-4f94-a982-acf1ddff3a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:30.614680 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 12/25] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jul 21 22:18:30 2022] POST /identity/v3/roles => generated 219 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:30.707873 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:30.755728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 14/26] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:30.841040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7d089-dc6c-48d1-a3a0-7c6f80958719 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:30.845639 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:30.969220 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7d089-dc6c-48d1-a3a0-7c6f80958719 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:31.010413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55b110-f5cb-456e-8b7f-397a70a882be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:31.011799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55b110-f5cb-456e-8b7f-397a70a882be None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.012302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55b110-f5cb-456e-8b7f-397a70a882be None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.012852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55b110-f5cb-456e-8b7f-397a70a882be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.052109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55b110-f5cb-456e-8b7f-397a70a882be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-2a55b110-f5cb-456e-8b7f-397a70a882be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:31.060103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:31.061099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:18:31.061867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 13/27] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:31.072339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21e317-afee-4f94-a982-acf1ddff3a73 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:31.089733 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e128e-7117-4510-a05c-8b22defadeac None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.090383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e128e-7117-4510-a05c-8b22defadeac None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.090916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e128e-7117-4510-a05c-8b22defadeac None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.093354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 14/28] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:31.099380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62b8c5-cf93-4c44-9b16-55707f857edf None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.099718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62b8c5-cf93-4c44-9b16-55707f857edf None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.099960 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62b8c5-cf93-4c44-9b16-55707f857edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.123650 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cd62b8c5-cf93-4c44-9b16-55707f857edf None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:31.125349 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cd62b8c5-cf93-4c44-9b16-55707f857edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:31.233023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21e317-afee-4f94-a982-acf1ddff3a73 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:31.240977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:31.256667 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 15/29] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:31.279866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9f012-b6be-4e51-b7ab-5dc5962f0424 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.280197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9f012-b6be-4e51-b7ab-5dc5962f0424 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.280422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9f012-b6be-4e51-b7ab-5dc5962f0424 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.281996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 16/30] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:31.288614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7892cc7-9a77-47b6-b74f-73620414a0a0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.288930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7892cc7-9a77-47b6-b74f-73620414a0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.289187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7892cc7-9a77-47b6-b74f-73620414a0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.303047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2b415-59db-4fb9-89e6-96ff3f273054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:31.304098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2b415-59db-4fb9-89e6-96ff3f273054 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.304618 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2b415-59db-4fb9-89e6-96ff3f273054 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.305103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2b415-59db-4fb9-89e6-96ff3f273054 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.306109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2b415-59db-4fb9-89e6-96ff3f273054 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:31.306628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2b415-59db-4fb9-89e6-96ff3f273054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:31.308408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2b415-59db-4fb9-89e6-96ff3f273054 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:31.310512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c7892cc7-9a77-47b6-b74f-73620414a0a0 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:31.311958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c7892cc7-9a77-47b6-b74f-73620414a0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:31.322981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 15/31] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:31.324173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2b415-59db-4fb9-89e6-96ff3f273054 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:31.370400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:31.403075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 17/32] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:31.412511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7892cc7-9a77-47b6-b74f-73620414a0a0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:31.481404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2b415-59db-4fb9-89e6-96ff3f273054 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:31.522964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b753608-7584-4605-ba97-d25591241128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:31.524391 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b753608-7584-4605-ba97-d25591241128 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.524915 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b753608-7584-4605-ba97-d25591241128 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.525414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b753608-7584-4605-ba97-d25591241128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.538003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b753608-7584-4605-ba97-d25591241128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:31.539826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b753608-7584-4605-ba97-d25591241128 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:31.541610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7892cc7-9a77-47b6-b74f-73620414a0a0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-7b753608-7584-4605-ba97-d25591241128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:18:31.568070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:31.569612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:31.571172 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:18:31.571172 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:18:31.571172 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:18:31.571172 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:18:31.572969 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:31.575321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 16/33] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:31.577272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b753608-7584-4605-ba97-d25591241128 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:31.587419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e3419-7400-4837-b0ee-48507dffb6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:31.588361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e3419-7400-4837-b0ee-48507dffb6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.589677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e3419-7400-4837-b0ee-48507dffb6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.589992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e3419-7400-4837-b0ee-48507dffb6d0 None admin] PATH_INFO: `/v3/users/e2841614037940fea030f1d6c1eca499` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.591365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e3419-7400-4837-b0ee-48507dffb6d0 None admin] RBAC: Authorizing `identity:get_user(user_id=e2841614037940fea030f1d6c1eca499)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:31.593115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e3419-7400-4837-b0ee-48507dffb6d0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:31.594822 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 18/34] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:31 2022] GET /identity/v3/users/e2841614037940fea030f1d6c1eca499 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:31.595627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e3419-7400-4837-b0ee-48507dffb6d0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:31.743008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:31.745891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e3419-7400-4837-b0ee-48507dffb6d0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:31.755906 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b753608-7584-4605-ba97-d25591241128 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:31.801789 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ceae1-16b0-4cc1-8977-69665e32562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:31.802804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ceae1-16b0-4cc1-8977-69665e32562f None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.803161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ceae1-16b0-4cc1-8977-69665e32562f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.803446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ceae1-16b0-4cc1-8977-69665e32562f None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.804294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ceae1-16b0-4cc1-8977-69665e32562f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:31.806303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ceae1-16b0-4cc1-8977-69665e32562f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:31.818953 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872319e-8178-45d4-a419-7a6e4493d0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:31.821610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872319e-8178-45d4-a419-7a6e4493d0af None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.822390 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872319e-8178-45d4-a419-7a6e4493d0af None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.823121 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872319e-8178-45d4-a419-7a6e4493d0af None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.825507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872319e-8178-45d4-a419-7a6e4493d0af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:31.826650 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872319e-8178-45d4-a419-7a6e4493d0af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:31.830043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872319e-8178-45d4-a419-7a6e4493d0af None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:31.869647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 19/35] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:31.871557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872319e-8178-45d4-a419-7a6e4493d0af None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:31.944588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 17/36] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jul 21 22:18:31 2022] POST /identity/v3/roles => generated 225 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:31.945387 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6c403-0db3-4b0a-81d4-87ba0cef63f8 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:31.945692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6c403-0db3-4b0a-81d4-87ba0cef63f8 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:31.945983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6c403-0db3-4b0a-81d4-87ba0cef63f8 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:31.947377 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 18/37] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:31.952133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6c403-0db3-4b0a-81d4-87ba0cef63f8 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:31.971745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:32.041571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872319e-8178-45d4-a419-7a6e4493d0af None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:32.084226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5c02e-7b06-429b-9659-01abf0178f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:32.084890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6c403-0db3-4b0a-81d4-87ba0cef63f8 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:32.086463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5c02e-7b06-429b-9659-01abf0178f7c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:32.087033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5c02e-7b06-429b-9659-01abf0178f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:32.087726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5c02e-7b06-429b-9659-01abf0178f7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:32.089485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5c02e-7b06-429b-9659-01abf0178f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:32.091424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5c02e-7b06-429b-9659-01abf0178f7c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:32.093441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 20/38] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:32.095049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab2ee8-155e-469e-bd48-f8734ed69f48 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:32.095576 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab2ee8-155e-469e-bd48-f8734ed69f48 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:32.096059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab2ee8-155e-469e-bd48-f8734ed69f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:32.115838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-45ab2ee8-155e-469e-bd48-f8734ed69f48 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:32.117630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-45ab2ee8-155e-469e-bd48-f8734ed69f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:32.133131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943c363-8c84-4dd2-8b7a-c2ed421b761a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:32.134293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943c363-8c84-4dd2-8b7a-c2ed421b761a None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:32.134729 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943c363-8c84-4dd2-8b7a-c2ed421b761a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:32.135149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943c363-8c84-4dd2-8b7a-c2ed421b761a None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:32.137060 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943c363-8c84-4dd2-8b7a-c2ed421b761a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:32.140820 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943c363-8c84-4dd2-8b7a-c2ed421b761a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:32.196599 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 19/39] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:18:31 2022] POST /identity/v3/projects => generated 301 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:32.198249 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943c363-8c84-4dd2-8b7a-c2ed421b761a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:32.202507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:32.231985 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 21/40] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:32.265770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab2ee8-155e-469e-bd48-f8734ed69f48 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:32.370683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943c363-8c84-4dd2-8b7a-c2ed421b761a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:32.423884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb338ed-1869-4cd6-80da-5cea276be7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:32.425860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb338ed-1869-4cd6-80da-5cea276be7d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:32.426240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb338ed-1869-4cd6-80da-5cea276be7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:32.426662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb338ed-1869-4cd6-80da-5cea276be7d6 None admin] PATH_INFO: `/v3/domains/default/users/e2841614037940fea030f1d6c1eca499/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:32.432826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb338ed-1869-4cd6-80da-5cea276be7d6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e2841614037940fea030f1d6c1eca499, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:32.434190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb338ed-1869-4cd6-80da-5cea276be7d6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:32.458652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 20/41] 10.222.0.42 () {58 vars in 1361 bytes} [Thu Jul 21 22:18:32 2022] PUT /identity/v3/domains/default/users/e2841614037940fea030f1d6c1eca499/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:32.540129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:32.637042 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab2ee8-155e-469e-bd48-f8734ed69f48 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:32.640409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed07cb-315c-4254-aae4-4eabca53efdd None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:32.640970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed07cb-315c-4254-aae4-4eabca53efdd None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:32.641493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed07cb-315c-4254-aae4-4eabca53efdd None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:32.645949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 21/42] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:32.652702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05276af7-6429-4971-a6a2-6959c2955cc8 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:32.653017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05276af7-6429-4971-a6a2-6959c2955cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:32.653724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05276af7-6429-4971-a6a2-6959c2955cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:32.672385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-05276af7-6429-4971-a6a2-6959c2955cc8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:32.674355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-05276af7-6429-4971-a6a2-6959c2955cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:32.686921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a16d0-eab9-4aa1-8d7e-98e092a35b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:32.687532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a16d0-eab9-4aa1-8d7e-98e092a35b56 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:32.687826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a16d0-eab9-4aa1-8d7e-98e092a35b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:32.688111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a16d0-eab9-4aa1-8d7e-98e092a35b56 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:32.688971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a16d0-eab9-4aa1-8d7e-98e092a35b56 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:32.691115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a16d0-eab9-4aa1-8d7e-98e092a35b56 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:32.707985 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 22/43] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jul 21 22:18:32 2022] POST /identity/v3/roles => generated 223 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:32.747586 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:32.768246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 22/44] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:32.793092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a16d0-eab9-4aa1-8d7e-98e092a35b56 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:32.935687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a16d0-eab9-4aa1-8d7e-98e092a35b56 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:32.952434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:32.982003 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e95e39-52fc-46fa-bcc5-ba893439906d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:32.982835 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e95e39-52fc-46fa-bcc5-ba893439906d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:32.983122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e95e39-52fc-46fa-bcc5-ba893439906d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:32.983415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e95e39-52fc-46fa-bcc5-ba893439906d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:32.984622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e95e39-52fc-46fa-bcc5-ba893439906d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:32.986111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e95e39-52fc-46fa-bcc5-ba893439906d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:32.988303 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 23/45] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:32.994039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05276af7-6429-4971-a6a2-6959c2955cc8 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:33.103547 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05276af7-6429-4971-a6a2-6959c2955cc8 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:33.142581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4233cb-b108-46ee-9f1a-6127f006087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:33.143350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4233cb-b108-46ee-9f1a-6127f006087f None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:33.144216 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4233cb-b108-46ee-9f1a-6127f006087f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:33.144622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4233cb-b108-46ee-9f1a-6127f006087f None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:33.145446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4233cb-b108-46ee-9f1a-6127f006087f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:33.147153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4233cb-b108-46ee-9f1a-6127f006087f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:33.178922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 23/46] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:18:32 2022] POST /identity/v3/projects => generated 312 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:18:35.752460 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e20143-08bd-48a5-9f96-116ee017eed1 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:35.753041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e20143-08bd-48a5-9f96-116ee017eed1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:35.753952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e20143-08bd-48a5-9f96-116ee017eed1 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:35.756880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 24/47] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:35.764696 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6674e3-a135-4408-90f1-e4f09e7048da None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:35.765461 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6674e3-a135-4408-90f1-e4f09e7048da None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:35.765753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6674e3-a135-4408-90f1-e4f09e7048da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:35.786078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-0c6674e3-a135-4408-90f1-e4f09e7048da None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:35.787301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0c6674e3-a135-4408-90f1-e4f09e7048da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:35.837820 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:35.870853 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 24/48] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:35.896164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e20143-08bd-48a5-9f96-116ee017eed1 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:36.012144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e20143-08bd-48a5-9f96-116ee017eed1 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:36.049207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6445cf-bf25-4264-a860-ed5530bb10a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:36.050716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6445cf-bf25-4264-a860-ed5530bb10a3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.051017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6445cf-bf25-4264-a860-ed5530bb10a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.051298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6445cf-bf25-4264-a860-ed5530bb10a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.063037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6445cf-bf25-4264-a860-ed5530bb10a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ff6445cf-bf25-4264-a860-ed5530bb10a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:36.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:36.072434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:18:36.074256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 25/49] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:36.078819 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6674e3-a135-4408-90f1-e4f09e7048da None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:36.191833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6674e3-a135-4408-90f1-e4f09e7048da None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:36.229882 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94956c29-8183-4970-9785-4f8c5a1a5e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:36.230585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94956c29-8183-4970-9785-4f8c5a1a5e5f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.230875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94956c29-8183-4970-9785-4f8c5a1a5e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.231159 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94956c29-8183-4970-9785-4f8c5a1a5e5f None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.231943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94956c29-8183-4970-9785-4f8c5a1a5e5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:36.232257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94956c29-8183-4970-9785-4f8c5a1a5e5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:36.233735 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94956c29-8183-4970-9785-4f8c5a1a5e5f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:36.242988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 25/50] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:36.248786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6445cf-bf25-4264-a860-ed5530bb10a3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:36.294337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:36.359278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6445cf-bf25-4264-a860-ed5530bb10a3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:36.396619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb92772-f0b9-4d97-a51b-538c5ad7d008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:36.397515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb92772-f0b9-4d97-a51b-538c5ad7d008 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.397803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb92772-f0b9-4d97-a51b-538c5ad7d008 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.398084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb92772-f0b9-4d97-a51b-538c5ad7d008 None admin] PATH_INFO: `/v3/users/e2841614037940fea030f1d6c1eca499` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.399595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb92772-f0b9-4d97-a51b-538c5ad7d008 None admin] RBAC: Authorizing `identity:get_user(user_id=e2841614037940fea030f1d6c1eca499)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:36.401122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb92772-f0b9-4d97-a51b-538c5ad7d008 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:36.402792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 26/51] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:36 2022] GET /identity/v3/users/e2841614037940fea030f1d6c1eca499 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:36.408499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94956c29-8183-4970-9785-4f8c5a1a5e5f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:36.546399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f0c2e-c5c9-4528-9156-112dcf043d0b None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.547164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f0c2e-c5c9-4528-9156-112dcf043d0b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.547537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f0c2e-c5c9-4528-9156-112dcf043d0b None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.549108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 27/52] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:36.555704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48f33-fe1a-40c5-9a46-be83afc2b8fe None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.556172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48f33-fe1a-40c5-9a46-be83afc2b8fe None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.557049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48f33-fe1a-40c5-9a46-be83afc2b8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.562251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:36.577212 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5ab48f33-fe1a-40c5-9a46-be83afc2b8fe None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:36.578762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5ab48f33-fe1a-40c5-9a46-be83afc2b8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:36.651806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94956c29-8183-4970-9785-4f8c5a1a5e5f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:36.672799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 28/53] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:36.694674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee7265-1439-4ffd-8b69-d36a70c84687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:36.695748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee7265-1439-4ffd-8b69-d36a70c84687 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.696156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee7265-1439-4ffd-8b69-d36a70c84687 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.697072 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee7265-1439-4ffd-8b69-d36a70c84687 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.698529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee7265-1439-4ffd-8b69-d36a70c84687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:36.700043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee7265-1439-4ffd-8b69-d36a70c84687 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:36.703626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 26/54] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:36.703847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48f33-fe1a-40c5-9a46-be83afc2b8fe None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:36.709864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee7265-1439-4ffd-8b69-d36a70c84687 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:36.748177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:36.832138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48f33-fe1a-40c5-9a46-be83afc2b8fe None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:36.873292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84f28e-7a44-426b-ac77-2fde24f2b244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:36.874272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84f28e-7a44-426b-ac77-2fde24f2b244 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.874642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84f28e-7a44-426b-ac77-2fde24f2b244 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.875005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84f28e-7a44-426b-ac77-2fde24f2b244 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.875939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee7265-1439-4ffd-8b69-d36a70c84687 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:36.876821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84f28e-7a44-426b-ac77-2fde24f2b244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:36.878762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84f28e-7a44-426b-ac77-2fde24f2b244 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:36.879374 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:36.881290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 29/55] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:36.890178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84f28e-7a44-426b-ac77-2fde24f2b244 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:36.930960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a39d9-61ce-4303-94f4-d979dbc5a828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:36.932005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a39d9-61ce-4303-94f4-d979dbc5a828 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:36.932520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a39d9-61ce-4303-94f4-d979dbc5a828 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:36.932986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a39d9-61ce-4303-94f4-d979dbc5a828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:36.934518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a39d9-61ce-4303-94f4-d979dbc5a828 None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.domain.id=default, user.id=e2841614037940fea030f1d6c1eca499) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:36.935050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a39d9-61ce-4303-94f4-d979dbc5a828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:36.937510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a39d9-61ce-4303-94f4-d979dbc5a828 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:36.964904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 27/56] 10.222.0.42 () {56 vars in 1322 bytes} [Thu Jul 21 22:18:36 2022] GET /identity/v3/role_assignments?user.id=e2841614037940fea030f1d6c1eca499&scope.domain.id=default&role.id=f957e09d047b47cfa0891d25b4181263 => generated 530 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:37.036724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84f28e-7a44-426b-ac77-2fde24f2b244 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:37.039942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:37.084176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbf300-22f2-4c4b-a4c9-fa32f91f020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:37.085053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbf300-22f2-4c4b-a4c9-fa32f91f020b None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:37.085492 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbf300-22f2-4c4b-a4c9-fa32f91f020b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:37.085899 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbf300-22f2-4c4b-a4c9-fa32f91f020b None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:37.087355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbf300-22f2-4c4b-a4c9-fa32f91f020b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:37.088398 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fbf300-22f2-4c4b-a4c9-fa32f91f020b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:37.126114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 30/57] 10.222.0.42 () {60 vars in 1140 bytes} [Thu Jul 21 22:18:36 2022] POST /identity/v3/projects => generated 298 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:18:39.343671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5d688-0b07-4d4b-b75b-4dcb9be74843 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:39.344268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5d688-0b07-4d4b-b75b-4dcb9be74843 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:39.344728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5d688-0b07-4d4b-b75b-4dcb9be74843 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:39.347799 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 28/58] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:39.353729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9abc2f-4e85-493c-813c-8a491c750a17 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:39.354245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9abc2f-4e85-493c-813c-8a491c750a17 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:39.354359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9abc2f-4e85-493c-813c-8a491c750a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:39.372421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1c9abc2f-4e85-493c-813c-8a491c750a17 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:39.373692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1c9abc2f-4e85-493c-813c-8a491c750a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:39.493811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 31/59] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:39.520520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5d688-0b07-4d4b-b75b-4dcb9be74843 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:39.637018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5d688-0b07-4d4b-b75b-4dcb9be74843 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:39.676305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a2074-ab11-461f-ab6a-e1b85f10a9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:39.677152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a2074-ab11-461f-ab6a-e1b85f10a9fe None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:39.677441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a2074-ab11-461f-ab6a-e1b85f10a9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:39.677722 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a2074-ab11-461f-ab6a-e1b85f10a9fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:39.678943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a2074-ab11-461f-ab6a-e1b85f10a9fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:39.680426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a2074-ab11-461f-ab6a-e1b85f10a9fe None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:39.682112 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 29/60] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:39.687941 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9abc2f-4e85-493c-813c-8a491c750a17 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:39.783949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:39.804856 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9abc2f-4e85-493c-813c-8a491c750a17 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:39.845469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983a24d-09ff-45ed-8339-497c39c22acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:39.846162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983a24d-09ff-45ed-8339-497c39c22acd None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:39.847541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983a24d-09ff-45ed-8339-497c39c22acd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:39.848010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983a24d-09ff-45ed-8339-497c39c22acd None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:39.848829 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983a24d-09ff-45ed-8339-497c39c22acd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:39.850373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983a24d-09ff-45ed-8339-497c39c22acd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:39.879819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 32/61] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:18:39 2022] POST /identity/v3/users => generated 275 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:18:44.029572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d4cff-edae-4bfd-a6f0-de59d2910ba2 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.030338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d4cff-edae-4bfd-a6f0-de59d2910ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.030338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d4cff-edae-4bfd-a6f0-de59d2910ba2 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.031777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 30/62] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:44.038094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2d43c-c900-4c3b-8bc9-7f537112a0d2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.039016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2d43c-c900-4c3b-8bc9-7f537112a0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.039494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2d43c-c900-4c3b-8bc9-7f537112a0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.065589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9da2d43c-c900-4c3b-8bc9-7f537112a0d2 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:44.067154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9da2d43c-c900-4c3b-8bc9-7f537112a0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:44.166950 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 33/63] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:44.202735 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d4cff-edae-4bfd-a6f0-de59d2910ba2 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.263947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:44.365056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d4cff-edae-4bfd-a6f0-de59d2910ba2 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:44.391124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae5d71-e432-475d-a46d-ea1f794f8d3f None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.391465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae5d71-e432-475d-a46d-ea1f794f8d3f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.391699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae5d71-e432-475d-a46d-ea1f794f8d3f None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.393169 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 34/64] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:44.405057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579884b-4d55-4af7-92aa-456d9543bb3c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.405680 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579884b-4d55-4af7-92aa-456d9543bb3c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.406434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579884b-4d55-4af7-92aa-456d9543bb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.430087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d9d8e-fc1c-4cb7-9084-2d83f0965114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:44.430087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d9d8e-fc1c-4cb7-9084-2d83f0965114 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.430087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d9d8e-fc1c-4cb7-9084-2d83f0965114 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.430087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d9d8e-fc1c-4cb7-9084-2d83f0965114 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.442290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3579884b-4d55-4af7-92aa-456d9543bb3c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:44.443509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3579884b-4d55-4af7-92aa-456d9543bb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:44.452265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d9d8e-fc1c-4cb7-9084-2d83f0965114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-424d9d8e-fc1c-4cb7-9084-2d83f0965114 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:44.461371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:44.462283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:18:44.463040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 31/65] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:18:44.481586 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d9d8e-fc1c-4cb7-9084-2d83f0965114 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.559959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 35/66] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:44.561559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9be7d-01b4-4555-ab7d-fa73457fd66d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.561858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9be7d-01b4-4555-ab7d-fa73457fd66d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.562142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9be7d-01b4-4555-ab7d-fa73457fd66d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.563480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 36/67] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:44.574308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830eb78-346e-43c4-a190-61bca528ae08 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.574929 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830eb78-346e-43c4-a190-61bca528ae08 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.575399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830eb78-346e-43c4-a190-61bca528ae08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.588855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:44.604598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9830eb78-346e-43c4-a190-61bca528ae08 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:44.606799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9830eb78-346e-43c4-a190-61bca528ae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:44.627053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:44.635031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d9d8e-fc1c-4cb7-9084-2d83f0965114 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:44.672546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356e147-de59-4b63-b305-321f7d9ad945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:44.673472 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356e147-de59-4b63-b305-321f7d9ad945 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.673903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356e147-de59-4b63-b305-321f7d9ad945 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.674352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356e147-de59-4b63-b305-321f7d9ad945 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.675303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356e147-de59-4b63-b305-321f7d9ad945 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:44.675770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356e147-de59-4b63-b305-321f7d9ad945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:44.677380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356e147-de59-4b63-b305-321f7d9ad945 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:44.686646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 32/68] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:44.692865 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356e147-de59-4b63-b305-321f7d9ad945 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.695043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 37/69] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:44.696278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830eb78-346e-43c4-a190-61bca528ae08 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.822876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830eb78-346e-43c4-a190-61bca528ae08 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:44.861962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356e147-de59-4b63-b305-321f7d9ad945 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:44.864107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:44.871082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b1730-58bf-4675-b49f-fdd34efe397e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:44.871978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b1730-58bf-4675-b49f-fdd34efe397e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.872263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b1730-58bf-4675-b49f-fdd34efe397e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.872734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b1730-58bf-4675-b49f-fdd34efe397e None admin] PATH_INFO: `/v3/users/e2841614037940fea030f1d6c1eca499` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.874113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b1730-58bf-4675-b49f-fdd34efe397e None admin] RBAC: Authorizing `identity:get_user(user_id=e2841614037940fea030f1d6c1eca499)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:44.875827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b1730-58bf-4675-b49f-fdd34efe397e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:44.877801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 38/70] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/users/e2841614037940fea030f1d6c1eca499 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:44.879761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b1730-58bf-4675-b49f-fdd34efe397e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.905117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d7248-5b3a-4c10-bc04-17aaf679351a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:44.909068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d7248-5b3a-4c10-bc04-17aaf679351a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.909068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d7248-5b3a-4c10-bc04-17aaf679351a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.909068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d7248-5b3a-4c10-bc04-17aaf679351a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.919912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d7248-5b3a-4c10-bc04-17aaf679351a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-ca5d7248-5b3a-4c10-bc04-17aaf679351a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:44.928814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:44.929843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:18:44.930860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 33/71] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:44.933858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c20df-e03e-415b-b280-bc537c485612 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.934163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c20df-e03e-415b-b280-bc537c485612 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.935636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c20df-e03e-415b-b280-bc537c485612 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.937131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 34/72] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:44.938399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28435c-30e6-4d3e-a540-c5bc4a47562c None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:44.938700 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28435c-30e6-4d3e-a540-c5bc4a47562c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:44.938987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28435c-30e6-4d3e-a540-c5bc4a47562c None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:44.941039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 35/73] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:44.941610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28435c-30e6-4d3e-a540-c5bc4a47562c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:44.964364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:45.011864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b1730-58bf-4675-b49f-fdd34efe397e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:45.069208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b148b-936e-4518-a372-083cb7a1b56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:45.070461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b148b-936e-4518-a372-083cb7a1b56c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.070922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b148b-936e-4518-a372-083cb7a1b56c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.071376 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b148b-936e-4518-a372-083cb7a1b56c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.082162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28435c-30e6-4d3e-a540-c5bc4a47562c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:45.092173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b148b-936e-4518-a372-083cb7a1b56c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-bd0b148b-936e-4518-a372-083cb7a1b56c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:45.107501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:45.108518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:45.109316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:45.109316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:45.109316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:45.109316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:45.109316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:45.109316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:45.109316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:45.109316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:45.109316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:45.109316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:45.109316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:45.109316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:18:45.109931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 39/74] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:45.111070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b148b-936e-4518-a372-083cb7a1b56c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:45.139836 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1197cf-7dd5-49ca-b651-7886e4d2d096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:45.140941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1197cf-7dd5-49ca-b651-7886e4d2d096 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.141448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1197cf-7dd5-49ca-b651-7886e4d2d096 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.141971 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1197cf-7dd5-49ca-b651-7886e4d2d096 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.189120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:45.241359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b148b-936e-4518-a372-083cb7a1b56c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:45.246243 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1197cf-7dd5-49ca-b651-7886e4d2d096 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:45.247772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1197cf-7dd5-49ca-b651-7886e4d2d096 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:45.268538 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:45.297471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e46d7-d390-4935-8719-262b0343306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:45.298641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e46d7-d390-4935-8719-262b0343306c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.299156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e46d7-d390-4935-8719-262b0343306c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.299567 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e46d7-d390-4935-8719-262b0343306c None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.301004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e46d7-d390-4935-8719-262b0343306c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:45.301455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e46d7-d390-4935-8719-262b0343306c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:45.303513 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e46d7-d390-4935-8719-262b0343306c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:45.314319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 40/75] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:18:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:45.317063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25defbfc-a272-4873-84b7-5c963411a9a2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.317411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25defbfc-a272-4873-84b7-5c963411a9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.317709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25defbfc-a272-4873-84b7-5c963411a9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.339981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-25defbfc-a272-4873-84b7-5c963411a9a2 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:45.341261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-25defbfc-a272-4873-84b7-5c963411a9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:45.384807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 36/76] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:45.386221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1a180-f96b-4b9d-840c-95ee4f086d30 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.386521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1a180-f96b-4b9d-840c-95ee4f086d30 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.386814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1a180-f96b-4b9d-840c-95ee4f086d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.407105 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-96c1a180-f96b-4b9d-840c-95ee4f086d30 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:45.408629 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-96c1a180-f96b-4b9d-840c-95ee4f086d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:45.448458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 41/77] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:45.453592 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25defbfc-a272-4873-84b7-5c963411a9a2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:45.514196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 37/78] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:45.516931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1a180-f96b-4b9d-840c-95ee4f086d30 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:45.564103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:45.589309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:45.631402 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25defbfc-a272-4873-84b7-5c963411a9a2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:45.662483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1a180-f96b-4b9d-840c-95ee4f086d30 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:45.675714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfe6a9-103b-4c8d-92d1-285ecafcd0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:45.676411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfe6a9-103b-4c8d-92d1-285ecafcd0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.676730 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfe6a9-103b-4c8d-92d1-285ecafcd0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.677011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfe6a9-103b-4c8d-92d1-285ecafcd0b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.677792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfe6a9-103b-4c8d-92d1-285ecafcd0b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:45.678097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfe6a9-103b-4c8d-92d1-285ecafcd0b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:45.679547 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfe6a9-103b-4c8d-92d1-285ecafcd0b4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:45.688855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 42/79] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:45.689785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfe6a9-103b-4c8d-92d1-285ecafcd0b4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:45.714244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416ef55-05ae-4999-b0d0-16da49323db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:45.715658 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416ef55-05ae-4999-b0d0-16da49323db9 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.716110 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416ef55-05ae-4999-b0d0-16da49323db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.716539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416ef55-05ae-4999-b0d0-16da49323db9 None admin] PATH_INFO: `/v3/users/67aea1a002a149b2aaa38b0dd4193f16` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.740290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416ef55-05ae-4999-b0d0-16da49323db9 None admin] RBAC: Authorizing `identity:get_user(user_id=67aea1a002a149b2aaa38b0dd4193f16)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:45.742231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416ef55-05ae-4999-b0d0-16da49323db9 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:45.744355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 38/80] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:45 2022] GET /identity/v3/users/67aea1a002a149b2aaa38b0dd4193f16 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:45.745311 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416ef55-05ae-4999-b0d0-16da49323db9 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:45.819493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfe6a9-103b-4c8d-92d1-285ecafcd0b4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:45.837382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:45.863888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4531cf-3877-4350-b7d1-b8a62cf66c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:45.864975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4531cf-3877-4350-b7d1-b8a62cf66c48 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.865365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4531cf-3877-4350-b7d1-b8a62cf66c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.865741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4531cf-3877-4350-b7d1-b8a62cf66c48 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.867236 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4531cf-3877-4350-b7d1-b8a62cf66c48 None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:45.868804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4531cf-3877-4350-b7d1-b8a62cf66c48 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:45.871651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 43/81] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:45 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:45.872902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4531cf-3877-4350-b7d1-b8a62cf66c48 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:45.880289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416ef55-05ae-4999-b0d0-16da49323db9 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:45.893962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:45.931564 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c33817-060c-4cdf-859c-dc00012451e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:45.932276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c33817-060c-4cdf-859c-dc00012451e3 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:45.932596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c33817-060c-4cdf-859c-dc00012451e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:45.932881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c33817-060c-4cdf-859c-dc00012451e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:45.945265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c33817-060c-4cdf-859c-dc00012451e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-a8c33817-060c-4cdf-859c-dc00012451e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:45.953279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:45.954228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:18:45.955268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 39/82] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:45.957193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c33817-060c-4cdf-859c-dc00012451e3 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:46.010914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4531cf-3877-4350-b7d1-b8a62cf66c48 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:46.052726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618719c-ca71-48a0-aeb1-a32999405f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:46.053660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618719c-ca71-48a0-aeb1-a32999405f67 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:46.054101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618719c-ca71-48a0-aeb1-a32999405f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:46.054475 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618719c-ca71-48a0-aeb1-a32999405f67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:46.055975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618719c-ca71-48a0-aeb1-a32999405f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:46.057637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618719c-ca71-48a0-aeb1-a32999405f67 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:46.059617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 44/83] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:46.060598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618719c-ca71-48a0-aeb1-a32999405f67 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:46.079671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c33817-060c-4cdf-859c-dc00012451e3 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:46.125016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e1364-f510-46e1-86eb-7ae6f3f2e6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:46.126392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e1364-f510-46e1-86eb-7ae6f3f2e6be None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:46.126726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e1364-f510-46e1-86eb-7ae6f3f2e6be None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:46.127011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e1364-f510-46e1-86eb-7ae6f3f2e6be None admin] PATH_INFO: `/v3/users/67aea1a002a149b2aaa38b0dd4193f16` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:46.131774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e1364-f510-46e1-86eb-7ae6f3f2e6be None admin] RBAC: Authorizing `identity:get_user(user_id=67aea1a002a149b2aaa38b0dd4193f16)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:46.132927 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:46.133611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e1364-f510-46e1-86eb-7ae6f3f2e6be None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:46.135635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 40/84] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:45 2022] GET /identity/v3/users/67aea1a002a149b2aaa38b0dd4193f16 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:46.136737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e1364-f510-46e1-86eb-7ae6f3f2e6be None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:46.180745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:46.204285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618719c-ca71-48a0-aeb1-a32999405f67 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:46.249752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171f640-43ff-4426-8d7a-8b11846bbce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:46.250666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171f640-43ff-4426-8d7a-8b11846bbce5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:46.251123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171f640-43ff-4426-8d7a-8b11846bbce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:46.251527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171f640-43ff-4426-8d7a-8b11846bbce5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:46.269929 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e1364-f510-46e1-86eb-7ae6f3f2e6be None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:46.320600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfca35-e585-4c38-aff8-d02ffb058988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:46.321670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfca35-e585-4c38-aff8-d02ffb058988 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:46.322156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfca35-e585-4c38-aff8-d02ffb058988 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:46.322623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfca35-e585-4c38-aff8-d02ffb058988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:46.324042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfca35-e585-4c38-aff8-d02ffb058988 None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=bd637189b43c45c28839d066df463bc3, user.id=e2841614037940fea030f1d6c1eca499) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:46.324601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfca35-e585-4c38-aff8-d02ffb058988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:46.327189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfca35-e585-4c38-aff8-d02ffb058988 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:46.348194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 41/85] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:46 2022] GET /identity/v3/role_assignments?user.id=e2841614037940fea030f1d6c1eca499&scope.project.id=bd637189b43c45c28839d066df463bc3&role.id=f957e09d047b47cfa0891d25b4181263 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:46.348194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfca35-e585-4c38-aff8-d02ffb058988 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:46.351892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171f640-43ff-4426-8d7a-8b11846bbce5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:46.354707 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171f640-43ff-4426-8d7a-8b11846bbce5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:46.441058 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:46.518479 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 45/86] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:46.519668 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171f640-43ff-4426-8d7a-8b11846bbce5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:46.530253 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:46.538905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfca35-e585-4c38-aff8-d02ffb058988 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:46.603745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769865e6-41af-46c4-b478-bf216319c98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:46.604767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769865e6-41af-46c4-b478-bf216319c98d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:46.605144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769865e6-41af-46c4-b478-bf216319c98d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:46.605569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769865e6-41af-46c4-b478-bf216319c98d None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:46.606739 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769865e6-41af-46c4-b478-bf216319c98d None admin] RBAC: Adding query filter params (name=member) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:46.607191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769865e6-41af-46c4-b478-bf216319c98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:46.608975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769865e6-41af-46c4-b478-bf216319c98d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:46.626634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 42/87] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:46.626634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769865e6-41af-46c4-b478-bf216319c98d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:46.672826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171f640-43ff-4426-8d7a-8b11846bbce5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:46.723243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246380-49a2-4fe3-b769-c862e1c5a2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:46.724119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246380-49a2-4fe3-b769-c862e1c5a2b5 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:46.724586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246380-49a2-4fe3-b769-c862e1c5a2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:46.725015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246380-49a2-4fe3-b769-c862e1c5a2b5 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:46.726224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1246380-49a2-4fe3-b769-c862e1c5a2b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:46.728135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1246380-49a2-4fe3-b769-c862e1c5a2b5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:46.754230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 46/88] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:18:46 2022] POST /identity/v3/users => generated 289 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:46.756188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246380-49a2-4fe3-b769-c862e1c5a2b5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:46.778480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:46.841059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769865e6-41af-46c4-b478-bf216319c98d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:46.912661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d1eec-d58b-4095-b20b-0d0dc05b1581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:46.914288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d1eec-d58b-4095-b20b-0d0dc05b1581 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:46.915295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d1eec-d58b-4095-b20b-0d0dc05b1581 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:46.915923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d1eec-d58b-4095-b20b-0d0dc05b1581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:46.957767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246380-49a2-4fe3-b769-c862e1c5a2b5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:46.974597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:47.026496 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f37f8-f764-41ce-bde4-51afd773cad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:47.028300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f37f8-f764-41ce-bde4-51afd773cad1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:47.029235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f37f8-f764-41ce-bde4-51afd773cad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:47.030052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f37f8-f764-41ce-bde4-51afd773cad1 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:47.032937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f37f8-f764-41ce-bde4-51afd773cad1 None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:47.036658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f37f8-f764-41ce-bde4-51afd773cad1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:47.040811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 47/89] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:46 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:47.042796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f37f8-f764-41ce-bde4-51afd773cad1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:47.054090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d1eec-d58b-4095-b20b-0d0dc05b1581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:47.056763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d1eec-d58b-4095-b20b-0d0dc05b1581 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:47.216622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:47.272939 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f37f8-f764-41ce-bde4-51afd773cad1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:47.286903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 43/90] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:47.288509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d1eec-d58b-4095-b20b-0d0dc05b1581 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:47.352183 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7dbbe-31e3-4689-a10b-df1a7c4f88bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:47.359908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7dbbe-31e3-4689-a10b-df1a7c4f88bd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:47.359908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7dbbe-31e3-4689-a10b-df1a7c4f88bd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:47.359908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7dbbe-31e3-4689-a10b-df1a7c4f88bd None admin] PATH_INFO: `/v3/users/67aea1a002a149b2aaa38b0dd4193f16` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:47.366054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7dbbe-31e3-4689-a10b-df1a7c4f88bd None admin] RBAC: Authorizing `identity:get_user(user_id=67aea1a002a149b2aaa38b0dd4193f16)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:47.366054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7dbbe-31e3-4689-a10b-df1a7c4f88bd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:47.368008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 48/91] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:47 2022] GET /identity/v3/users/67aea1a002a149b2aaa38b0dd4193f16 => generated 275 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:47.371677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7dbbe-31e3-4689-a10b-df1a7c4f88bd None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:47.444797 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:47.528724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d1eec-d58b-4095-b20b-0d0dc05b1581 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:47.577847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7dbbe-31e3-4689-a10b-df1a7c4f88bd None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:47.603294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a0a7e-947b-457a-96b5-5deb080b8ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:47.604952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a0a7e-947b-457a-96b5-5deb080b8ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:47.605790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a0a7e-947b-457a-96b5-5deb080b8ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:47.606563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a0a7e-947b-457a-96b5-5deb080b8ff4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:47.608537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a0a7e-947b-457a-96b5-5deb080b8ff4 None admin] RBAC: Adding query filter params (role.id=80eea410deca46e2924f27afe5136d55, scope.project.id=bd637189b43c45c28839d066df463bc3, user.id=67aea1a002a149b2aaa38b0dd4193f16) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:47.613525 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a0a7e-947b-457a-96b5-5deb080b8ff4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:47.616565 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a0a7e-947b-457a-96b5-5deb080b8ff4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:47.639082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 44/92] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:47 2022] GET /identity/v3/role_assignments?user.id=67aea1a002a149b2aaa38b0dd4193f16&scope.project.id=bd637189b43c45c28839d066df463bc3&role.id=80eea410deca46e2924f27afe5136d55 => generated 259 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:47.641266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a0a7e-947b-457a-96b5-5deb080b8ff4 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:47.708952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4da0e-ba11-403e-9638-48015efc64d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:47.711106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4da0e-ba11-403e-9638-48015efc64d4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:47.712290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4da0e-ba11-403e-9638-48015efc64d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:47.713090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4da0e-ba11-403e-9638-48015efc64d4 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:47.720512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4da0e-ba11-403e-9638-48015efc64d4 None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:47.724796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4da0e-ba11-403e-9638-48015efc64d4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:47.728904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 49/93] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:47 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:47.758655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4da0e-ba11-403e-9638-48015efc64d4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:47.803637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:47.894041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a0a7e-947b-457a-96b5-5deb080b8ff4 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:47.989043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:48.004510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2db71-a11a-4b00-9c83-6b8611f036d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:48.005210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2db71-a11a-4b00-9c83-6b8611f036d0 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.005505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2db71-a11a-4b00-9c83-6b8611f036d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.006897 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2db71-a11a-4b00-9c83-6b8611f036d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:48.062320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4da0e-ba11-403e-9638-48015efc64d4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:48.175233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2db71-a11a-4b00-9c83-6b8611f036d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:48.176671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2db71-a11a-4b00-9c83-6b8611f036d0 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:48.182353 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92547d-1323-4249-b5d6-034349bf0b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:48.183466 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92547d-1323-4249-b5d6-034349bf0b19 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.183980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92547d-1323-4249-b5d6-034349bf0b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.184503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92547d-1323-4249-b5d6-034349bf0b19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:48.186286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92547d-1323-4249-b5d6-034349bf0b19 None admin] RBAC: Adding query filter params (role.id=1b870be20b0c4159bfbbe73593799250, scope.project.id=bd637189b43c45c28839d066df463bc3, user.id=67aea1a002a149b2aaa38b0dd4193f16) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:48.186842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92547d-1323-4249-b5d6-034349bf0b19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:48.190461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92547d-1323-4249-b5d6-034349bf0b19 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:48.208365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 50/94] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:47 2022] GET /identity/v3/role_assignments?user.id=67aea1a002a149b2aaa38b0dd4193f16&scope.project.id=bd637189b43c45c28839d066df463bc3&role.id=1b870be20b0c4159bfbbe73593799250 => generated 259 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:48.393892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 45/95] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 753 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:48.402174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92547d-1323-4249-b5d6-034349bf0b19 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:48.589078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:48.647498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92547d-1323-4249-b5d6-034349bf0b19 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:48.716139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1137ce8-30e6-40af-bf61-2b13b1d9c58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:48.718176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1137ce8-30e6-40af-bf61-2b13b1d9c58c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:48.718954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1137ce8-30e6-40af-bf61-2b13b1d9c58c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:48.719710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1137ce8-30e6-40af-bf61-2b13b1d9c58c None admin] PATH_INFO: `/v3/projects/c172869d2c004a73be21e86aca1e9213` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:48.722906 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1137ce8-30e6-40af-bf61-2b13b1d9c58c None admin] RBAC: Authorizing `identity:get_project(project_id=c172869d2c004a73be21e86aca1e9213)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:48.726405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1137ce8-30e6-40af-bf61-2b13b1d9c58c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:48.730453 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 51/96] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:48 2022] GET /identity/v3/projects/c172869d2c004a73be21e86aca1e9213 => generated 312 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:48.742883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2db71-a11a-4b00-9c83-6b8611f036d0 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:48.808945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:48.997436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2db71-a11a-4b00-9c83-6b8611f036d0 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:49.093995 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f1524-84d5-4f4e-bdd9-ffee2c47a5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:49.096281 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f1524-84d5-4f4e-bdd9-ffee2c47a5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:49.097173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f1524-84d5-4f4e-bdd9-ffee2c47a5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:49.097998 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f1524-84d5-4f4e-bdd9-ffee2c47a5d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:49.100285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f1524-84d5-4f4e-bdd9-ffee2c47a5d9 None admin] RBAC: Adding query filter params (role.id=1b870be20b0c4159bfbbe73593799250, scope.project.id=c172869d2c004a73be21e86aca1e9213, user.id=67aea1a002a149b2aaa38b0dd4193f16) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:49.101287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f1524-84d5-4f4e-bdd9-ffee2c47a5d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:49.104293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f1524-84d5-4f4e-bdd9-ffee2c47a5d9 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:49.120851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 46/97] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:48 2022] GET /identity/v3/role_assignments?user.id=67aea1a002a149b2aaa38b0dd4193f16&scope.project.id=c172869d2c004a73be21e86aca1e9213&role.id=1b870be20b0c4159bfbbe73593799250 => generated 259 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:49.721404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f146c7-8e14-49e8-ad58-9d82000064ed None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:49.722419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f146c7-8e14-49e8-ad58-9d82000064ed None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:49.722948 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f146c7-8e14-49e8-ad58-9d82000064ed None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:49.724657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 52/98] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:49.742025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79da08-01b1-4e6b-92ee-a41c90ecb048 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:49.742995 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79da08-01b1-4e6b-92ee-a41c90ecb048 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:49.743530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79da08-01b1-4e6b-92ee-a41c90ecb048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:49.765602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7a79da08-01b1-4e6b-92ee-a41c90ecb048 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:49.766834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7a79da08-01b1-4e6b-92ee-a41c90ecb048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:49.853622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 47/99] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:49.894863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f146c7-8e14-49e8-ad58-9d82000064ed None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:50.052154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f146c7-8e14-49e8-ad58-9d82000064ed None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:50.105250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7cba0-ffcb-4f8b-88ca-dda24dff6a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:50.106121 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7cba0-ffcb-4f8b-88ca-dda24dff6a4c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:50.106481 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7cba0-ffcb-4f8b-88ca-dda24dff6a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:50.106830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7cba0-ffcb-4f8b-88ca-dda24dff6a4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:50.129922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7cba0-ffcb-4f8b-88ca-dda24dff6a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-22f7cba0-ffcb-4f8b-88ca-dda24dff6a4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:50.138057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:50.138987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:18:50.139754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 53/100] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:50.149240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79da08-01b1-4e6b-92ee-a41c90ecb048 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:50.172601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:50.282771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79da08-01b1-4e6b-92ee-a41c90ecb048 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:50.333349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b8aeb-60fa-44b1-b6d2-032712a88ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:50.333965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b8aeb-60fa-44b1-b6d2-032712a88ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:50.334254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b8aeb-60fa-44b1-b6d2-032712a88ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:50.334532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b8aeb-60fa-44b1-b6d2-032712a88ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:50.335314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b8aeb-60fa-44b1-b6d2-032712a88ff4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:50.335625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b8aeb-60fa-44b1-b6d2-032712a88ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:50.337061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b8aeb-60fa-44b1-b6d2-032712a88ff4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:50.351029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 48/101] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:50.357053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7cba0-ffcb-4f8b-88ca-dda24dff6a4c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:50.360732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:50.521881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7cba0-ffcb-4f8b-88ca-dda24dff6a4c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:50.568851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d432ea-3415-4e74-848a-1032def3c812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:50.569770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d432ea-3415-4e74-848a-1032def3c812 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:50.570059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d432ea-3415-4e74-848a-1032def3c812 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:50.570352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d432ea-3415-4e74-848a-1032def3c812 None admin] PATH_INFO: `/v3/users/e2841614037940fea030f1d6c1eca499` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:50.571696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d432ea-3415-4e74-848a-1032def3c812 None admin] RBAC: Authorizing `identity:get_user(user_id=e2841614037940fea030f1d6c1eca499)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:50.573220 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d432ea-3415-4e74-848a-1032def3c812 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:50.574943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 54/102] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:50 2022] GET /identity/v3/users/e2841614037940fea030f1d6c1eca499 => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:50.584196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b8aeb-60fa-44b1-b6d2-032712a88ff4 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:50.729149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:50.764017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b8aeb-60fa-44b1-b6d2-032712a88ff4 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:50.822942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45fe11-a7e7-4c35-afbc-7cbd4fdeaf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:50.823868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45fe11-a7e7-4c35-afbc-7cbd4fdeaf9f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:50.824158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45fe11-a7e7-4c35-afbc-7cbd4fdeaf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:50.824442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45fe11-a7e7-4c35-afbc-7cbd4fdeaf9f None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:50.825875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45fe11-a7e7-4c35-afbc-7cbd4fdeaf9f None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:50.827386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be45fe11-a7e7-4c35-afbc-7cbd4fdeaf9f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:50.829327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 49/103] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:50 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:50.836630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d432ea-3415-4e74-848a-1032def3c812 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:50.981044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:51.031049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d432ea-3415-4e74-848a-1032def3c812 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:51.099854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7c152-1447-425a-85a9-4e7b5e192bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:51.100723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c152-1447-425a-85a9-4e7b5e192bbc None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.101014 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c152-1447-425a-85a9-4e7b5e192bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.101299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c152-1447-425a-85a9-4e7b5e192bbc None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3/users/e2841614037940fea030f1d6c1eca499/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.106039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7c152-1447-425a-85a9-4e7b5e192bbc None admin] RBAC: Authorizing `identity:create_grant(project_id=bd637189b43c45c28839d066df463bc3, user_id=e2841614037940fea030f1d6c1eca499, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:51.107550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7c152-1447-425a-85a9-4e7b5e192bbc None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:51.129656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 55/104] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:50 2022] PUT /identity/v3/projects/bd637189b43c45c28839d066df463bc3/users/e2841614037940fea030f1d6c1eca499/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:51.197589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c2d87-2602-4726-bf4a-2c8f44611a6f None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.198968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c2d87-2602-4726-bf4a-2c8f44611a6f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.199280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c2d87-2602-4726-bf4a-2c8f44611a6f None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.200840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 50/105] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:51.207274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff90467-2d24-4b6c-bbea-109f534e8081 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.207579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff90467-2d24-4b6c-bbea-109f534e8081 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.207854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff90467-2d24-4b6c-bbea-109f534e8081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.233732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cff90467-2d24-4b6c-bbea-109f534e8081 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:51.234980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cff90467-2d24-4b6c-bbea-109f534e8081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:51.243973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fae59-d848-46c2-b845-cbe1c3329599 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.244279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fae59-d848-46c2-b845-cbe1c3329599 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.244571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fae59-d848-46c2-b845-cbe1c3329599 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.246133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 51/106] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:18:51.253309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdddf94-7cf4-4ce3-ad10-9cbbb2fd966c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.253793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdddf94-7cf4-4ce3-ad10-9cbbb2fd966c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.254083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdddf94-7cf4-4ce3-ad10-9cbbb2fd966c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.282752 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ecdddf94-7cf4-4ce3-ad10-9cbbb2fd966c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:51.284066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ecdddf94-7cf4-4ce3-ad10-9cbbb2fd966c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:51.420741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 56/107] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:51.474394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 52/108] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:18:51.501434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff90467-2d24-4b6c-bbea-109f534e8081 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:51.520073 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdddf94-7cf4-4ce3-ad10-9cbbb2fd966c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:51.668940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:51.733224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:51.744154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff90467-2d24-4b6c-bbea-109f534e8081 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:51.803092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdddf94-7cf4-4ce3-ad10-9cbbb2fd966c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:51.864873 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f33871-e009-467a-96bb-1b3929623f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:51.866376 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f33871-e009-467a-96bb-1b3929623f5d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.866873 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f33871-e009-467a-96bb-1b3929623f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.867353 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f33871-e009-467a-96bb-1b3929623f5d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.890332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893f6c0-6fb3-4700-8562-b26f7b25d300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:51.891655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893f6c0-6fb3-4700-8562-b26f7b25d300 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:51.892134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893f6c0-6fb3-4700-8562-b26f7b25d300 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:51.901038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893f6c0-6fb3-4700-8562-b26f7b25d300 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:51.903504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893f6c0-6fb3-4700-8562-b26f7b25d300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:51.906363 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893f6c0-6fb3-4700-8562-b26f7b25d300 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:51.908182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f33871-e009-467a-96bb-1b3929623f5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:51.914760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 53/109] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-28f33871-e009-467a-96bb-1b3929623f5d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:51.928619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:51.929654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:51.930418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:51.930418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:51.930418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:51.930418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:51.930418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:51.930418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:51.930418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:51.930418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:51.930418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:51.930418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:51.930418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:51.930418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:18:51.932922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 57/110] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:51.941123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f33871-e009-467a-96bb-1b3929623f5d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:51.941630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893f6c0-6fb3-4700-8562-b26f7b25d300 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:52.171324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f33871-e009-467a-96bb-1b3929623f5d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:52.213925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893f6c0-6fb3-4700-8562-b26f7b25d300 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:52.224925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:52.253484 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ab8db-ed1a-4b51-9ebc-98f73e8d4866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:52.255242 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ab8db-ed1a-4b51-9ebc-98f73e8d4866 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.256005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ab8db-ed1a-4b51-9ebc-98f73e8d4866 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.256732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ab8db-ed1a-4b51-9ebc-98f73e8d4866 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:52.258378 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ab8db-ed1a-4b51-9ebc-98f73e8d4866 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:52.261911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ab8db-ed1a-4b51-9ebc-98f73e8d4866 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:52.279776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27dca35-930e-4c48-bdd6-abeaf0c10557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:52.281219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dca35-930e-4c48-bdd6-abeaf0c10557 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.281719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dca35-930e-4c48-bdd6-abeaf0c10557 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.282355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dca35-930e-4c48-bdd6-abeaf0c10557 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:52.284700 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dca35-930e-4c48-bdd6-abeaf0c10557 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:52.285540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dca35-930e-4c48-bdd6-abeaf0c10557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:52.288106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dca35-930e-4c48-bdd6-abeaf0c10557 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:52.310631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 54/111] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:52.313607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7602c4-a17e-4a9b-85f6-3acc818bc704 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.314487 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7602c4-a17e-4a9b-85f6-3acc818bc704 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.316748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7602c4-a17e-4a9b-85f6-3acc818bc704 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:52.319698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 55/112] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:52.321041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 58/113] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:18:51 2022] POST /identity/v3/projects => generated 302 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:52.322817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7602c4-a17e-4a9b-85f6-3acc818bc704 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:52.330396 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94cab7-c2c7-43c9-bd15-6955e6743b88 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.330935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94cab7-c2c7-43c9-bd15-6955e6743b88 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.331701 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94cab7-c2c7-43c9-bd15-6955e6743b88 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:52.336720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 59/114] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:52 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:52.344514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689a1a5-9997-47c2-ad93-d001fb49cebe None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.346582 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689a1a5-9997-47c2-ad93-d001fb49cebe None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.347331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689a1a5-9997-47c2-ad93-d001fb49cebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:52.370478 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:52.416813 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e689a1a5-9997-47c2-ad93-d001fb49cebe None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:52.418995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e689a1a5-9997-47c2-ad93-d001fb49cebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:52.603122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7602c4-a17e-4a9b-85f6-3acc818bc704 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:52.624876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 60/115] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:52.627670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcb839-df44-40e4-938f-a26ef1e11f1a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.628124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcb839-df44-40e4-938f-a26ef1e11f1a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.630870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcb839-df44-40e4-938f-a26ef1e11f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:52.667022 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1ddcb839-df44-40e4-938f-a26ef1e11f1a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:52.669075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1ddcb839-df44-40e4-938f-a26ef1e11f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:52.678944 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113180c5-5179-4fec-b5bc-6afb63f898c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:52.681061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113180c5-5179-4fec-b5bc-6afb63f898c0 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.681903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113180c5-5179-4fec-b5bc-6afb63f898c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.682869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113180c5-5179-4fec-b5bc-6afb63f898c0 None admin] PATH_INFO: `/v3/users/ba4f45224476433ea9ae59fb6650ccad` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:52.713111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113180c5-5179-4fec-b5bc-6afb63f898c0 None admin] RBAC: Authorizing `identity:get_user(user_id=ba4f45224476433ea9ae59fb6650ccad)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:52.716619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113180c5-5179-4fec-b5bc-6afb63f898c0 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:52.720514 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 56/116] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:52 2022] GET /identity/v3/users/ba4f45224476433ea9ae59fb6650ccad => generated 289 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:52.722502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113180c5-5179-4fec-b5bc-6afb63f898c0 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:52.792532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 61/117] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:52.792532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4c587-93d9-42d4-a448-b5fab8ba7a8b None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.792532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4c587-93d9-42d4-a448-b5fab8ba7a8b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.792532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4c587-93d9-42d4-a448-b5fab8ba7a8b None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:52.800805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 62/118] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:52.801804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4c587-93d9-42d4-a448-b5fab8ba7a8b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:52.849383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:52.894648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:52.920577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113180c5-5179-4fec-b5bc-6afb63f898c0 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:52.994176 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032114af-ab38-4e17-b666-d2b117eb1a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:52.995996 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032114af-ab38-4e17-b666-d2b117eb1a34 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:52.997734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032114af-ab38-4e17-b666-d2b117eb1a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:52.997734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032114af-ab38-4e17-b666-d2b117eb1a34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.020997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032114af-ab38-4e17-b666-d2b117eb1a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-032114af-ab38-4e17-b666-d2b117eb1a34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:53.039437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:53.041273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:53.042715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:53.042715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:53.042715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:53.042715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:53.042715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:53.042715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:53.042715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:53.042715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:53.042715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:53.042715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:53.042715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:18:53.042715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:18:53.046341 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 57/119] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:18:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:53.049810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c1cb-e512-4ee3-a365-ec48f877fb03 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:53.050107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c1cb-e512-4ee3-a365-ec48f877fb03 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:53.050750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c1cb-e512-4ee3-a365-ec48f877fb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.071920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4c587-93d9-42d4-a448-b5fab8ba7a8b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:53.083270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-c814c1cb-e512-4ee3-a365-ec48f877fb03 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:53.085654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c814c1cb-e512-4ee3-a365-ec48f877fb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:53.160075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d6030-7cea-4a68-bedd-b311ab94b2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:53.163246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d6030-7cea-4a68-bedd-b311ab94b2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:53.164048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d6030-7cea-4a68-bedd-b311ab94b2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:53.164808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d6030-7cea-4a68-bedd-b311ab94b2a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.231887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 58/120] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:53.233844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c1cb-e512-4ee3-a365-ec48f877fb03 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:53.358451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d6030-7cea-4a68-bedd-b311ab94b2a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:53.361942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d6030-7cea-4a68-bedd-b311ab94b2a2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:53.376245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:53.444102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:53.471711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c1cb-e512-4ee3-a365-ec48f877fb03 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:53.522097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dad92-f75c-46d1-ad79-715e22aa8a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:53.523470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dad92-f75c-46d1-ad79-715e22aa8a1e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:53.523843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dad92-f75c-46d1-ad79-715e22aa8a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:53.524201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dad92-f75c-46d1-ad79-715e22aa8a1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.538592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0dad92-f75c-46d1-ad79-715e22aa8a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-fe0dad92-f75c-46d1-ad79-715e22aa8a1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:53.547130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:53.549740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:18:53.551712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 59/121] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:53.562454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dad92-f75c-46d1-ad79-715e22aa8a1e None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:53.604643 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 63/122] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 803 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:18:53.606658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d6030-7cea-4a68-bedd-b311ab94b2a2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:53.680993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dad92-f75c-46d1-ad79-715e22aa8a1e None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:53.700878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:53.725757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2316e-2a74-476e-a19e-cefc59d85b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:53.726729 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2316e-2a74-476e-a19e-cefc59d85b90 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:53.727198 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2316e-2a74-476e-a19e-cefc59d85b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:53.727624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2316e-2a74-476e-a19e-cefc59d85b90 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.728594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2316e-2a74-476e-a19e-cefc59d85b90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:53.729055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2316e-2a74-476e-a19e-cefc59d85b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:53.730731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2316e-2a74-476e-a19e-cefc59d85b90 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:53.741738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 60/123] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:18:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:53.743090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2316e-2a74-476e-a19e-cefc59d85b90 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:53.809333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d6030-7cea-4a68-bedd-b311ab94b2a2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:53.870325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2316e-2a74-476e-a19e-cefc59d85b90 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:53.871369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:53.880825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac95962-e026-4924-966f-a69ecbee4f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:53.882026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac95962-e026-4924-966f-a69ecbee4f89 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:53.882568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac95962-e026-4924-966f-a69ecbee4f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:53.883050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac95962-e026-4924-966f-a69ecbee4f89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.901752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac95962-e026-4924-966f-a69ecbee4f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-bac95962-e026-4924-966f-a69ecbee4f89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:53.915501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:53.916449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:53.917586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:53.917586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:53.917586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:53.917586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:53.917586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:53.917586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:53.917586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:53.917586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:53.917586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:53.917586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:53.917586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:18:53.917586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:18:53.924142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 64/124] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:53.924142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac95962-e026-4924-966f-a69ecbee4f89 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:53.925054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3dd1c8-ff7b-4c42-a746-0b7aa10ef74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:53.925770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3dd1c8-ff7b-4c42-a746-0b7aa10ef74a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:53.926060 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3dd1c8-ff7b-4c42-a746-0b7aa10ef74a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:53.926340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3dd1c8-ff7b-4c42-a746-0b7aa10ef74a None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:53.927154 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3dd1c8-ff7b-4c42-a746-0b7aa10ef74a None admin] RBAC: Adding query filter params (name=member) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:53.927466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3dd1c8-ff7b-4c42-a746-0b7aa10ef74a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:53.928960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3dd1c8-ff7b-4c42-a746-0b7aa10ef74a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:53.939595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 61/125] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:53.940094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3dd1c8-ff7b-4c42-a746-0b7aa10ef74a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:53.998401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:54.065650 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac95962-e026-4924-966f-a69ecbee4f89 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:54.105123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3dd1c8-ff7b-4c42-a746-0b7aa10ef74a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:54.106244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1e6e7-383f-417a-929d-0e076806d4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:54.107166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1e6e7-383f-417a-929d-0e076806d4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.107479 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1e6e7-383f-417a-929d-0e076806d4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.107728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1e6e7-383f-417a-929d-0e076806d4f8 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.109094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1e6e7-383f-417a-929d-0e076806d4f8 None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:54.110592 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1e6e7-383f-417a-929d-0e076806d4f8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:54.112972 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 65/126] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:53 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:54.118167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90584da-a639-42bc-9fb5-53db027e6ed0 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.118167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90584da-a639-42bc-9fb5-53db027e6ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.118167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90584da-a639-42bc-9fb5-53db027e6ed0 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.120373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 66/127] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:54.121253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90584da-a639-42bc-9fb5-53db027e6ed0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:54.168352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7eca8-55db-4cf2-9b77-b5d7772b8dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:54.169854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7eca8-55db-4cf2-9b77-b5d7772b8dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.170340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7eca8-55db-4cf2-9b77-b5d7772b8dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.170769 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7eca8-55db-4cf2-9b77-b5d7772b8dd4 None admin] PATH_INFO: `/v3/users/67aea1a002a149b2aaa38b0dd4193f16` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.173140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7eca8-55db-4cf2-9b77-b5d7772b8dd4 None admin] RBAC: Authorizing `identity:get_user(user_id=67aea1a002a149b2aaa38b0dd4193f16)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:54.176579 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7eca8-55db-4cf2-9b77-b5d7772b8dd4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:54.184464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 62/128] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:53 2022] GET /identity/v3/users/67aea1a002a149b2aaa38b0dd4193f16 => generated 275 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:54.184464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7eca8-55db-4cf2-9b77-b5d7772b8dd4 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:54.200593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:54.247430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90584da-a639-42bc-9fb5-53db027e6ed0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:54.286286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31782a41-732b-4107-96c9-b856b4a11c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:54.286968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31782a41-732b-4107-96c9-b856b4a11c1c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.287254 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31782a41-732b-4107-96c9-b856b4a11c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.287537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31782a41-732b-4107-96c9-b856b4a11c1c None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.288294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31782a41-732b-4107-96c9-b856b4a11c1c None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:54.288638 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31782a41-732b-4107-96c9-b856b4a11c1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:54.290050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31782a41-732b-4107-96c9-b856b4a11c1c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:54.305503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 67/129] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:54.307048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31782a41-732b-4107-96c9-b856b4a11c1c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:54.336847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7eca8-55db-4cf2-9b77-b5d7772b8dd4 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:54.346763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:54.373031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568a510-289a-4a7e-ad25-b2864f0bf38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:54.373925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568a510-289a-4a7e-ad25-b2864f0bf38a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.374214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568a510-289a-4a7e-ad25-b2864f0bf38a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.374495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568a510-289a-4a7e-ad25-b2864f0bf38a None admin] PATH_INFO: `/v3/users/67aea1a002a149b2aaa38b0dd4193f16` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.375758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568a510-289a-4a7e-ad25-b2864f0bf38a None admin] RBAC: Authorizing `identity:get_user(user_id=67aea1a002a149b2aaa38b0dd4193f16)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:54.377758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568a510-289a-4a7e-ad25-b2864f0bf38a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:54.379880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 63/130] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:54 2022] GET /identity/v3/users/67aea1a002a149b2aaa38b0dd4193f16 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:54.384604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fc0a0-7bd7-4826-b816-a5f22890b180 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.384604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fc0a0-7bd7-4826-b816-a5f22890b180 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.384604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2fc0a0-7bd7-4826-b816-a5f22890b180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.411613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-bf2fc0a0-7bd7-4826-b816-a5f22890b180 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:54.417874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bf2fc0a0-7bd7-4826-b816-a5f22890b180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:54.494148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31782a41-732b-4107-96c9-b856b4a11c1c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:54.532061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:54.541555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 64/131] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:54.542922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2fc0a0-7bd7-4826-b816-a5f22890b180 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:54.568835 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b73df6-7c11-4894-9646-77c9e9ac673f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:54.570688 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73df6-7c11-4894-9646-77c9e9ac673f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.572024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73df6-7c11-4894-9646-77c9e9ac673f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.572862 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b73df6-7c11-4894-9646-77c9e9ac673f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.574676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b73df6-7c11-4894-9646-77c9e9ac673f None admin] RBAC: Adding query filter params (role.id=35dbb86a4b774e0a9d862b6fa21cd947, scope.project.id=bd637189b43c45c28839d066df463bc3, user.id=ba4f45224476433ea9ae59fb6650ccad) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:54.575504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b73df6-7c11-4894-9646-77c9e9ac673f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:54.579525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b73df6-7c11-4894-9646-77c9e9ac673f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:54.590878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 68/132] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:54 2022] GET /identity/v3/role_assignments?user.id=ba4f45224476433ea9ae59fb6650ccad&scope.project.id=bd637189b43c45c28839d066df463bc3&role.id=35dbb86a4b774e0a9d862b6fa21cd947 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:54.604850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b73df6-7c11-4894-9646-77c9e9ac673f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:54.726790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2fc0a0-7bd7-4826-b816-a5f22890b180 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:54.741409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:54.795098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033c6a-b664-481f-a0e9-76f7b83a0a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:54.796540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033c6a-b664-481f-a0e9-76f7b83a0a5f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.797028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033c6a-b664-481f-a0e9-76f7b83a0a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.797511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033c6a-b664-481f-a0e9-76f7b83a0a5f None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.799718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00033c6a-b664-481f-a0e9-76f7b83a0a5f None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:54.802221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00033c6a-b664-481f-a0e9-76f7b83a0a5f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:54.805400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 65/133] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:54 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:54.811992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b73df6-7c11-4894-9646-77c9e9ac673f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:54.814490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033c6a-b664-481f-a0e9-76f7b83a0a5f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:54.883536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746326f-db38-41fb-99cc-20e8451a7b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:54.885406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746326f-db38-41fb-99cc-20e8451a7b11 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:54.886122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746326f-db38-41fb-99cc-20e8451a7b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:54.886843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746326f-db38-41fb-99cc-20e8451a7b11 None admin] PATH_INFO: `/v3/users/67aea1a002a149b2aaa38b0dd4193f16` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:54.890005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746326f-db38-41fb-99cc-20e8451a7b11 None admin] RBAC: Authorizing `identity:get_user(user_id=67aea1a002a149b2aaa38b0dd4193f16)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:54.894472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746326f-db38-41fb-99cc-20e8451a7b11 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:54.898997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 69/134] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:54 2022] GET /identity/v3/users/67aea1a002a149b2aaa38b0dd4193f16 => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:54.901012 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746326f-db38-41fb-99cc-20e8451a7b11 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.010173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:55.019415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033c6a-b664-481f-a0e9-76f7b83a0a5f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.086600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e9428-2bbc-409b-9b44-e31338ae0115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:55.088204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e9428-2bbc-409b-9b44-e31338ae0115 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.088983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e9428-2bbc-409b-9b44-e31338ae0115 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.089483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e9428-2bbc-409b-9b44-e31338ae0115 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.092678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e9428-2bbc-409b-9b44-e31338ae0115 None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:55.097163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e9428-2bbc-409b-9b44-e31338ae0115 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:55.101275 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 66/135] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:54 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:55.102731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e9428-2bbc-409b-9b44-e31338ae0115 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.111985 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746326f-db38-41fb-99cc-20e8451a7b11 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.173786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d402a4-4c2d-4dee-97c4-f9c90221d562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:55.174609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d402a4-4c2d-4dee-97c4-f9c90221d562 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.174889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d402a4-4c2d-4dee-97c4-f9c90221d562 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.175236 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d402a4-4c2d-4dee-97c4-f9c90221d562 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.180244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:55.187861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d402a4-4c2d-4dee-97c4-f9c90221d562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-f6d402a4-4c2d-4dee-97c4-f9c90221d562 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:55.197470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:55.199124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 70/136] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:18:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:55.201558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d402a4-4c2d-4dee-97c4-f9c90221d562 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.284355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e9428-2bbc-409b-9b44-e31338ae0115 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.346147 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64d606-1d13-4d10-b7a8-a80f5cd872df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:55.347562 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64d606-1d13-4d10-b7a8-a80f5cd872df None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.348028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64d606-1d13-4d10-b7a8-a80f5cd872df None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.348532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64d606-1d13-4d10-b7a8-a80f5cd872df None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3/users/67aea1a002a149b2aaa38b0dd4193f16/roles/80eea410deca46e2924f27afe5136d55` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.352196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64d606-1d13-4d10-b7a8-a80f5cd872df None admin] RBAC: Authorizing `identity:create_grant(project_id=bd637189b43c45c28839d066df463bc3, user_id=67aea1a002a149b2aaa38b0dd4193f16, role_id=80eea410deca46e2924f27afe5136d55)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:55.354724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64d606-1d13-4d10-b7a8-a80f5cd872df None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:55.373171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d402a4-4c2d-4dee-97c4-f9c90221d562 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.378452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 67/137] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:55 2022] PUT /identity/v3/projects/bd637189b43c45c28839d066df463bc3/users/67aea1a002a149b2aaa38b0dd4193f16/roles/80eea410deca46e2924f27afe5136d55 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:55.379975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64d606-1d13-4d10-b7a8-a80f5cd872df None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.406828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:55.467914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579c7c8-bf3b-4d4a-98d6-fb9393ceebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:55.469949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579c7c8-bf3b-4d4a-98d6-fb9393ceebc5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.470921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579c7c8-bf3b-4d4a-98d6-fb9393ceebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.471744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579c7c8-bf3b-4d4a-98d6-fb9393ceebc5 None admin] PATH_INFO: `/v3/projects/c172869d2c004a73be21e86aca1e9213` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.475565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579c7c8-bf3b-4d4a-98d6-fb9393ceebc5 None admin] RBAC: Authorizing `identity:get_project(project_id=c172869d2c004a73be21e86aca1e9213)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:55.479792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579c7c8-bf3b-4d4a-98d6-fb9393ceebc5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:55.485201 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 71/138] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:55 2022] GET /identity/v3/projects/c172869d2c004a73be21e86aca1e9213 => generated 312 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:55.492518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2b14f-ef4c-4290-951e-3d48ee8f3061 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.492951 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2b14f-ef4c-4290-951e-3d48ee8f3061 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.493458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2b14f-ef4c-4290-951e-3d48ee8f3061 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.495976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 72/139] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:55.497327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2b14f-ef4c-4290-951e-3d48ee8f3061 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.621886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64d606-1d13-4d10-b7a8-a80f5cd872df None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.623962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:55.695225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7ba4e-966b-46d5-82ee-a71d84c01afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:55.697063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7ba4e-966b-46d5-82ee-a71d84c01afb None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.697944 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7ba4e-966b-46d5-82ee-a71d84c01afb None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.698194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7ba4e-966b-46d5-82ee-a71d84c01afb None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3/users/67aea1a002a149b2aaa38b0dd4193f16/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.702053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7ba4e-966b-46d5-82ee-a71d84c01afb None admin] RBAC: Authorizing `identity:create_grant(project_id=bd637189b43c45c28839d066df463bc3, user_id=67aea1a002a149b2aaa38b0dd4193f16, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:55.705860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7ba4e-966b-46d5-82ee-a71d84c01afb None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:55.716952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2b14f-ef4c-4290-951e-3d48ee8f3061 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.733898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 68/140] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:55 2022] PUT /identity/v3/projects/bd637189b43c45c28839d066df463bc3/users/67aea1a002a149b2aaa38b0dd4193f16/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:55.735140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7ba4e-966b-46d5-82ee-a71d84c01afb None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:55.797632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876dbc9-106b-458d-b2dd-f323c056aebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:55.800317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876dbc9-106b-458d-b2dd-f323c056aebe None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.801220 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876dbc9-106b-458d-b2dd-f323c056aebe None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.801997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876dbc9-106b-458d-b2dd-f323c056aebe None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.803839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876dbc9-106b-458d-b2dd-f323c056aebe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:55.804766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876dbc9-106b-458d-b2dd-f323c056aebe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:55.807956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876dbc9-106b-458d-b2dd-f323c056aebe None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:55.827801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 73/141] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:55.834355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffca63-a2d5-4650-bc7a-73e3ea635efd None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:55.835362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffca63-a2d5-4650-bc7a-73e3ea635efd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:55.836009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffca63-a2d5-4650-bc7a-73e3ea635efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:55.879973 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f9ffca63-a2d5-4650-bc7a-73e3ea635efd None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:55.882700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f9ffca63-a2d5-4650-bc7a-73e3ea635efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:55.969173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7ba4e-966b-46d5-82ee-a71d84c01afb None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:55.993653 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:56.044773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82379f37-4da6-478b-954a-5d750eb902af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:56.047414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82379f37-4da6-478b-954a-5d750eb902af None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.048189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82379f37-4da6-478b-954a-5d750eb902af None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.049606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82379f37-4da6-478b-954a-5d750eb902af None admin] PATH_INFO: `/v3/projects/c172869d2c004a73be21e86aca1e9213/users/67aea1a002a149b2aaa38b0dd4193f16/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.053926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82379f37-4da6-478b-954a-5d750eb902af None admin] RBAC: Authorizing `identity:create_grant(project_id=c172869d2c004a73be21e86aca1e9213, user_id=67aea1a002a149b2aaa38b0dd4193f16, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:56.057249 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82379f37-4da6-478b-954a-5d750eb902af None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:56.058501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 74/142] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:56.060011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffca63-a2d5-4650-bc7a-73e3ea635efd None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:56.094033 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 69/143] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:55 2022] PUT /identity/v3/projects/c172869d2c004a73be21e86aca1e9213/users/67aea1a002a149b2aaa38b0dd4193f16/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:18:56.136977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82379f37-4da6-478b-954a-5d750eb902af None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:56.220979 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:56.339942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffca63-a2d5-4650-bc7a-73e3ea635efd None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:56.389036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82379f37-4da6-478b-954a-5d750eb902af None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:56.402095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fce7e8-37fa-4867-acb1-802e2a2e45af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:56.403533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fce7e8-37fa-4867-acb1-802e2a2e45af None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.403997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fce7e8-37fa-4867-acb1-802e2a2e45af None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.404442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fce7e8-37fa-4867-acb1-802e2a2e45af None admin] PATH_INFO: `/v3/users/e2841614037940fea030f1d6c1eca499` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.406752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fce7e8-37fa-4867-acb1-802e2a2e45af None admin] RBAC: Authorizing `identity:get_user(user_id=e2841614037940fea030f1d6c1eca499)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:56.409516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fce7e8-37fa-4867-acb1-802e2a2e45af None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:56.412610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 75/144] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:56 2022] GET /identity/v3/users/e2841614037940fea030f1d6c1eca499 => generated 247 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:56.422237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fce7e8-37fa-4867-acb1-802e2a2e45af None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:56.498781 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b89826-77da-4ec0-a8d8-b707b95a579d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:56.500196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b89826-77da-4ec0-a8d8-b707b95a579d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.502494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b89826-77da-4ec0-a8d8-b707b95a579d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.503036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b89826-77da-4ec0-a8d8-b707b95a579d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.506041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b89826-77da-4ec0-a8d8-b707b95a579d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:56.514951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b89826-77da-4ec0-a8d8-b707b95a579d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:56.519703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 70/145] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:18:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:56.528731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b89826-77da-4ec0-a8d8-b707b95a579d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:56.571329 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:56.737740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:56.760388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b89826-77da-4ec0-a8d8-b707b95a579d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:56.795845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fce7e8-37fa-4867-acb1-802e2a2e45af None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:56.836705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e19daa-76f7-4300-a650-04d9c3bd619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:56.837914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e19daa-76f7-4300-a650-04d9c3bd619e None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.838440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e19daa-76f7-4300-a650-04d9c3bd619e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.838953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e19daa-76f7-4300-a650-04d9c3bd619e None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:56.840515 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e19daa-76f7-4300-a650-04d9c3bd619e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:56.843327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e19daa-76f7-4300-a650-04d9c3bd619e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:56.911917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 71/146] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:18:56 2022] POST /identity/v3/users => generated 283 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:18:56.917149 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff055ed-6844-4695-9dd9-edc877cc9b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:56.918859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff055ed-6844-4695-9dd9-edc877cc9b12 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:56.919400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff055ed-6844-4695-9dd9-edc877cc9b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:56.919883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff055ed-6844-4695-9dd9-edc877cc9b12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:57.071374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff055ed-6844-4695-9dd9-edc877cc9b12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:57.074224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff055ed-6844-4695-9dd9-edc877cc9b12 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:57.173945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:57.313511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 76/147] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:18:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 892 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:18:57.321111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e19daa-76f7-4300-a650-04d9c3bd619e None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:57.547371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e19daa-76f7-4300-a650-04d9c3bd619e None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:57.619736 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1c7b9-4b56-4987-b636-51a5d039bbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:57.621275 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1c7b9-4b56-4987-b636-51a5d039bbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:57.621774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1c7b9-4b56-4987-b636-51a5d039bbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:57.622257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1c7b9-4b56-4987-b636-51a5d039bbd2 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:57.624547 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1c7b9-4b56-4987-b636-51a5d039bbd2 None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:57.628092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1c7b9-4b56-4987-b636-51a5d039bbd2 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:57.632193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 72/148] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:57 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:57.638584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff055ed-6844-4695-9dd9-edc877cc9b12 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:57.872098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff055ed-6844-4695-9dd9-edc877cc9b12 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:57.943719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd0031-c964-4635-9e2e-2f7cb0cd4acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:57.944819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd0031-c964-4635-9e2e-2f7cb0cd4acc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:57.945301 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd0031-c964-4635-9e2e-2f7cb0cd4acc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:57.945779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd0031-c964-4635-9e2e-2f7cb0cd4acc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:57.947442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd0031-c964-4635-9e2e-2f7cb0cd4acc None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=bd637189b43c45c28839d066df463bc3, user.id=e2841614037940fea030f1d6c1eca499) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:57.947994 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd0031-c964-4635-9e2e-2f7cb0cd4acc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:57.950656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd0031-c964-4635-9e2e-2f7cb0cd4acc None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:57.963870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 77/149] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:18:57 2022] GET /identity/v3/role_assignments?user.id=e2841614037940fea030f1d6c1eca499&scope.project.id=bd637189b43c45c28839d066df463bc3&role.id=f957e09d047b47cfa0891d25b4181263 => generated 608 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:58.008382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5de0b9-ba8e-4d43-b2c4-b11176b7bed0 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:58.008886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5de0b9-ba8e-4d43-b2c4-b11176b7bed0 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:58.009764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5de0b9-ba8e-4d43-b2c4-b11176b7bed0 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:58.012192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 73/150] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:58.020736 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa9c05-a91c-453f-a56b-4b2775a5b67e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:58.021282 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa9c05-a91c-453f-a56b-4b2775a5b67e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:58.021941 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa9c05-a91c-453f-a56b-4b2775a5b67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:58.048025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:58.067297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2efa9c05-a91c-453f-a56b-4b2775a5b67e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:58.069633 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2efa9c05-a91c-453f-a56b-4b2775a5b67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:18:58.243242 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 78/151] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:18:58.283776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5de0b9-ba8e-4d43-b2c4-b11176b7bed0 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:58.318619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:58.527616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5de0b9-ba8e-4d43-b2c4-b11176b7bed0 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:58.619354 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449be8d1-9bcf-400b-a149-65cc53d3f438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:58.620712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449be8d1-9bcf-400b-a149-65cc53d3f438 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:58.621220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449be8d1-9bcf-400b-a149-65cc53d3f438 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:58.624511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449be8d1-9bcf-400b-a149-65cc53d3f438 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:58.659603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449be8d1-9bcf-400b-a149-65cc53d3f438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-449be8d1-9bcf-400b-a149-65cc53d3f438 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:18:58.681775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:18:58.683462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:18:58.684987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 74/152] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:18:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:18:58.692463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa9c05-a91c-453f-a56b-4b2775a5b67e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:58.879380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa9c05-a91c-453f-a56b-4b2775a5b67e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:58.940231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6f60a-2307-438a-8003-f8baadadaedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:58.940961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6f60a-2307-438a-8003-f8baadadaedc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:58.941265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6f60a-2307-438a-8003-f8baadadaedc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:58.941502 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6f60a-2307-438a-8003-f8baadadaedc None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:58.942278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6f60a-2307-438a-8003-f8baadadaedc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:18:58.942642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6f60a-2307-438a-8003-f8baadadaedc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:58.944076 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6f60a-2307-438a-8003-f8baadadaedc None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:58.953546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 79/153] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:18:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:18:58.955128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:58.963609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449be8d1-9bcf-400b-a149-65cc53d3f438 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:59.192956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:18:59.298769 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449be8d1-9bcf-400b-a149-65cc53d3f438 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:59.460070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383258b5-a6fb-40f5-affe-a45827e225d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:59.461568 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383258b5-a6fb-40f5-affe-a45827e225d7 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:59.462076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383258b5-a6fb-40f5-affe-a45827e225d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:59.462538 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383258b5-a6fb-40f5-affe-a45827e225d7 None admin] PATH_INFO: `/v3/users/ba4f45224476433ea9ae59fb6650ccad` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:59.469760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383258b5-a6fb-40f5-affe-a45827e225d7 None admin] RBAC: Authorizing `identity:get_user(user_id=ba4f45224476433ea9ae59fb6650ccad)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:59.473016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383258b5-a6fb-40f5-affe-a45827e225d7 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:59.476389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 75/154] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:18:58 2022] GET /identity/v3/users/ba4f45224476433ea9ae59fb6650ccad => generated 289 bytes in 514 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:59.486294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6f60a-2307-438a-8003-f8baadadaedc None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:59.675168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6f60a-2307-438a-8003-f8baadadaedc None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:18:59.754234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0f2b4-6bcf-4309-a628-ec30f65ce44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:18:59.755685 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f2b4-6bcf-4309-a628-ec30f65ce44d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:59.756166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f2b4-6bcf-4309-a628-ec30f65ce44d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:59.756686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f2b4-6bcf-4309-a628-ec30f65ce44d None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:59.761893 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0f2b4-6bcf-4309-a628-ec30f65ce44d None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:18:59.764773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0f2b4-6bcf-4309-a628-ec30f65ce44d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:18:59.768800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 80/155] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:18:59 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:18:59.778619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383258b5-a6fb-40f5-affe-a45827e225d7 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:18:59.800548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf18e3b-c264-4556-827d-dc9059a5ab16 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:59.800548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf18e3b-c264-4556-827d-dc9059a5ab16 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:59.800548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf18e3b-c264-4556-827d-dc9059a5ab16 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:59.801405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 81/156] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:18:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:18:59.813391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea92823-b856-47f4-b32e-f63a2c8f9082 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:18:59.814554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea92823-b856-47f4-b32e-f63a2c8f9082 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:18:59.815050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea92823-b856-47f4-b32e-f63a2c8f9082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:18:59.874270 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0ea92823-b856-47f4-b32e-f63a2c8f9082 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:18:59.878504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0ea92823-b856-47f4-b32e-f63a2c8f9082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:00.113702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383258b5-a6fb-40f5-affe-a45827e225d7 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:00.117187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:00.148930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 82/157] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 21 22:19:00.200234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d19898-d7a0-44fe-a4f3-942e5a029685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:00.201548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d19898-d7a0-44fe-a4f3-942e5a029685 None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:00.202040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d19898-d7a0-44fe-a4f3-942e5a029685 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:00.202535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d19898-d7a0-44fe-a4f3-942e5a029685 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3/users/ba4f45224476433ea9ae59fb6650ccad/roles/35dbb86a4b774e0a9d862b6fa21cd947` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:00.210075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d19898-d7a0-44fe-a4f3-942e5a029685 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd637189b43c45c28839d066df463bc3, user_id=ba4f45224476433ea9ae59fb6650ccad, role_id=35dbb86a4b774e0a9d862b6fa21cd947)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:00.211892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d19898-d7a0-44fe-a4f3-942e5a029685 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:00.225275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea92823-b856-47f4-b32e-f63a2c8f9082 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:00.238775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 76/158] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:18:59 2022] PUT /identity/v3/projects/bd637189b43c45c28839d066df463bc3/users/ba4f45224476433ea9ae59fb6650ccad/roles/35dbb86a4b774e0a9d862b6fa21cd947 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:00.483220 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea92823-b856-47f4-b32e-f63a2c8f9082 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:00.560644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da51a7e-306c-417b-b397-83fb77f94bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:00.562037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da51a7e-306c-417b-b397-83fb77f94bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:00.562534 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da51a7e-306c-417b-b397-83fb77f94bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:00.563036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da51a7e-306c-417b-b397-83fb77f94bc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:00.585057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da51a7e-306c-417b-b397-83fb77f94bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-5da51a7e-306c-417b-b397-83fb77f94bc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:00.601842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:00.602904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:00.603683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:00.603683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:00.603683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:00.603683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:00.603683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:00.603683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:00.603683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:00.603683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:00.603683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:00.603683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:00.603683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:00.603683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:00.605018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 83/159] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:00.613690 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d19898-d7a0-44fe-a4f3-942e5a029685 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:00.620659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503daf50-c824-4df4-adae-02466455591d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:00.621359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:00.621545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503daf50-c824-4df4-adae-02466455591d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:00.621681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503daf50-c824-4df4-adae-02466455591d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:00.624164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 84/160] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:00.633991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd93e26-26e1-4d4b-9d46-30f38fd3d13f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:00.634459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd93e26-26e1-4d4b-9d46-30f38fd3d13f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:00.634911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd93e26-26e1-4d4b-9d46-30f38fd3d13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:00.686395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2fd93e26-26e1-4d4b-9d46-30f38fd3d13f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:00.688425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2fd93e26-26e1-4d4b-9d46-30f38fd3d13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:00.841319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:00.871105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 85/161] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:00.896737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d19898-d7a0-44fe-a4f3-942e5a029685 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:01.000920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ae787-9da1-43ab-ae80-7ec18b2a2273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:01.002612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ae787-9da1-43ab-ae80-7ec18b2a2273 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.003112 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ae787-9da1-43ab-ae80-7ec18b2a2273 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.003619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ae787-9da1-43ab-ae80-7ec18b2a2273 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.004834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd93e26-26e1-4d4b-9d46-30f38fd3d13f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:01.005140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ae787-9da1-43ab-ae80-7ec18b2a2273 None admin] RBAC: Adding query filter params (name=member) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:01.005794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ae787-9da1-43ab-ae80-7ec18b2a2273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:01.008546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ae787-9da1-43ab-ae80-7ec18b2a2273 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:01.028643 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 77/162] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:01.042389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ae787-9da1-43ab-ae80-7ec18b2a2273 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:01.249036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:01.293428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd93e26-26e1-4d4b-9d46-30f38fd3d13f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:01.366666 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ae787-9da1-43ab-ae80-7ec18b2a2273 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:01.397182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6df36-3f35-4cbb-aba8-04904fe4934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:01.398537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6df36-3f35-4cbb-aba8-04904fe4934f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.398963 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6df36-3f35-4cbb-aba8-04904fe4934f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.399391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6df36-3f35-4cbb-aba8-04904fe4934f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.412798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6df36-3f35-4cbb-aba8-04904fe4934f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-43d6df36-3f35-4cbb-aba8-04904fe4934f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:01.422004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:01.423604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:01.425168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:01.425168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:01.425168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:01.425168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:01.425168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:01.425168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:01.425168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:01.425168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:01.425168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:01.425168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:01.425168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:01.425168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:01.427250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 86/163] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:01.428859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bac4f9-a9dc-4e15-b246-20c5072c9f9a None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.429297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bac4f9-a9dc-4e15-b246-20c5072c9f9a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.429732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bac4f9-a9dc-4e15-b246-20c5072c9f9a None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.431516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 87/164] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:01.438384 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7e0ab-125c-41bf-84f7-3c5b974dd278 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.439578 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7e0ab-125c-41bf-84f7-3c5b974dd278 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.440327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7e0ab-125c-41bf-84f7-3c5b974dd278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.441945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55f164-6f5a-4136-9988-72e3900058e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:01.442868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55f164-6f5a-4136-9988-72e3900058e0 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.443162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55f164-6f5a-4136-9988-72e3900058e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.443463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55f164-6f5a-4136-9988-72e3900058e0 None admin] PATH_INFO: `/v3/users/67aea1a002a149b2aaa38b0dd4193f16` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.455332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55f164-6f5a-4136-9988-72e3900058e0 None admin] RBAC: Authorizing `identity:get_user(user_id=67aea1a002a149b2aaa38b0dd4193f16)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:01.457239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55f164-6f5a-4136-9988-72e3900058e0 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:01.464596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 78/165] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:01 2022] GET /identity/v3/users/67aea1a002a149b2aaa38b0dd4193f16 => generated 275 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:01.465618 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55f164-6f5a-4136-9988-72e3900058e0 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:01.466190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d8c7e0ab-125c-41bf-84f7-3c5b974dd278 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:01.467583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d8c7e0ab-125c-41bf-84f7-3c5b974dd278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:01.538849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:01.598403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 88/166] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:01.605556 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7e0ab-125c-41bf-84f7-3c5b974dd278 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:01.780286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55f164-6f5a-4136-9988-72e3900058e0 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:01.811993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7e0ab-125c-41bf-84f7-3c5b974dd278 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:01.865714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:01.873070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1944cd-e714-4b21-ab3b-9580d8e7d302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:01.873784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1944cd-e714-4b21-ab3b-9580d8e7d302 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.874090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1944cd-e714-4b21-ab3b-9580d8e7d302 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.874340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1944cd-e714-4b21-ab3b-9580d8e7d302 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:01.875123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1944cd-e714-4b21-ab3b-9580d8e7d302 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:01.875434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1944cd-e714-4b21-ab3b-9580d8e7d302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:01.877806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1944cd-e714-4b21-ab3b-9580d8e7d302 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:01.888037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 79/167] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:01.888703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1944cd-e714-4b21-ab3b-9580d8e7d302 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:01.888987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc89abb-3929-498f-a819-b656c4455afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:01.890081 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc89abb-3929-498f-a819-b656c4455afa None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:01.890568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc89abb-3929-498f-a819-b656c4455afa None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:01.891047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc89abb-3929-498f-a819-b656c4455afa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.034640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1944cd-e714-4b21-ab3b-9580d8e7d302 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.068036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:02.089085 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aeeccc-417d-447f-a206-a78214a12612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.091031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aeeccc-417d-447f-a206-a78214a12612 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.092185 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aeeccc-417d-447f-a206-a78214a12612 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.092668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aeeccc-417d-447f-a206-a78214a12612 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.107370 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aeeccc-417d-447f-a206-a78214a12612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-17aeeccc-417d-447f-a206-a78214a12612 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:02.116394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:02.117325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:02.118109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:02.118109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:02.118109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:02.118109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:02.118109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:02.118109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:02.118109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:02.118109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:02.118109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:02.118109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:02.118109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:02.118109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:02.120732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 80/168] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:02.122270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b327f-2162-46ce-86c4-06c13ae0d650 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.127772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b327f-2162-46ce-86c4-06c13ae0d650 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.128293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b327f-2162-46ce-86c4-06c13ae0d650 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.130266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 81/169] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:02.131218 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b327f-2162-46ce-86c4-06c13ae0d650 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.134786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc89abb-3929-498f-a819-b656c4455afa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:02.144765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc89abb-3929-498f-a819-b656c4455afa None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:02.201615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:02.252171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b327f-2162-46ce-86c4-06c13ae0d650 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.315259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c410526-8a5e-46eb-ba13-2a567d49d5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.316544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c410526-8a5e-46eb-ba13-2a567d49d5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.317004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c410526-8a5e-46eb-ba13-2a567d49d5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.317449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c410526-8a5e-46eb-ba13-2a567d49d5f7 None admin] PATH_INFO: `/v3/users/67aea1a002a149b2aaa38b0dd4193f16` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.318925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c410526-8a5e-46eb-ba13-2a567d49d5f7 None admin] RBAC: Authorizing `identity:get_user(user_id=67aea1a002a149b2aaa38b0dd4193f16)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:02.320741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c410526-8a5e-46eb-ba13-2a567d49d5f7 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:02.322682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 82/170] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/users/67aea1a002a149b2aaa38b0dd4193f16 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:02.324206 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c410526-8a5e-46eb-ba13-2a567d49d5f7 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.440908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c410526-8a5e-46eb-ba13-2a567d49d5f7 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.466897 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:02.481675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d53263-976b-4955-98a0-f19c510a7eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.482721 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d53263-976b-4955-98a0-f19c510a7eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.483177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d53263-976b-4955-98a0-f19c510a7eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.483657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d53263-976b-4955-98a0-f19c510a7eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.484630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d53263-976b-4955-98a0-f19c510a7eb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:02.485141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d53263-976b-4955-98a0-f19c510a7eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:02.487406 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d53263-976b-4955-98a0-f19c510a7eb1 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:02.497135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 83/171] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:02.498698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032abd58-7788-4f53-83cc-28ede32aa248 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.499163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032abd58-7788-4f53-83cc-28ede32aa248 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.499641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032abd58-7788-4f53-83cc-28ede32aa248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.503877 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 89/172] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 904 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:02.504784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc89abb-3929-498f-a819-b656c4455afa None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.518715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-032abd58-7788-4f53-83cc-28ede32aa248 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:02.520296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-032abd58-7788-4f53-83cc-28ede32aa248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:02.599334 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 84/173] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:02.600572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032abd58-7788-4f53-83cc-28ede32aa248 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:02.659997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:02.685910 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc89abb-3929-498f-a819-b656c4455afa None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.726203 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a92a42-49c3-41eb-a5d1-174702edf799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.727199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a92a42-49c3-41eb-a5d1-174702edf799 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.727725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a92a42-49c3-41eb-a5d1-174702edf799 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.728147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a92a42-49c3-41eb-a5d1-174702edf799 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.787026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032abd58-7788-4f53-83cc-28ede32aa248 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:02.813154 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:02.851253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a92a42-49c3-41eb-a5d1-174702edf799 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:02.852781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a92a42-49c3-41eb-a5d1-174702edf799 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:02.855437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aaa0e-74cf-44eb-b961-0d47703ae42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:02.856335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aaa0e-74cf-44eb-b961-0d47703ae42b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:02.856646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aaa0e-74cf-44eb-b961-0d47703ae42b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:02.856925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aaa0e-74cf-44eb-b961-0d47703ae42b None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:02.859092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aaa0e-74cf-44eb-b961-0d47703ae42b None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:02.862173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aaa0e-74cf-44eb-b961-0d47703ae42b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:02.864036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 85/174] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:02.865005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aaa0e-74cf-44eb-b961-0d47703ae42b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.053431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:03.131457 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 90/175] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:03.134572 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a92a42-49c3-41eb-a5d1-174702edf799 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.144299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aaa0e-74cf-44eb-b961-0d47703ae42b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.207059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec56a07-f987-49b4-9549-600f0cb9927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.208665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec56a07-f987-49b4-9549-600f0cb9927d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.209214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec56a07-f987-49b4-9549-600f0cb9927d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.209777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec56a07-f987-49b4-9549-600f0cb9927d None admin] PATH_INFO: `/v3/users/67aea1a002a149b2aaa38b0dd4193f16` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.211746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec56a07-f987-49b4-9549-600f0cb9927d None admin] RBAC: Authorizing `identity:get_user(user_id=67aea1a002a149b2aaa38b0dd4193f16)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.214403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec56a07-f987-49b4-9549-600f0cb9927d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.217402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 86/176] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:02 2022] GET /identity/v3/users/67aea1a002a149b2aaa38b0dd4193f16 => generated 275 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:03.218951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec56a07-f987-49b4-9549-600f0cb9927d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.299401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:03.325150 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a92a42-49c3-41eb-a5d1-174702edf799 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.372447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec56a07-f987-49b4-9549-600f0cb9927d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.384029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa24be-9c27-4bf4-8776-9b16a7de1423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.385215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa24be-9c27-4bf4-8776-9b16a7de1423 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.385662 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa24be-9c27-4bf4-8776-9b16a7de1423 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.386091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa24be-9c27-4bf4-8776-9b16a7de1423 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.404898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa24be-9c27-4bf4-8776-9b16a7de1423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-5baa24be-9c27-4bf4-8776-9b16a7de1423 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:03.418134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:03.419119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 91/177] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:03.425824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78226e09-183c-44b0-848b-79d47eb424d6 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.426563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78226e09-183c-44b0-848b-79d47eb424d6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.426563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78226e09-183c-44b0-848b-79d47eb424d6 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.426563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 92/178] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:03.426563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78226e09-183c-44b0-848b-79d47eb424d6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.434203 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1989ec-9646-44cc-b05f-6c0ca1b46437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.435171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1989ec-9646-44cc-b05f-6c0ca1b46437 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.435683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1989ec-9646-44cc-b05f-6c0ca1b46437 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.436034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1989ec-9646-44cc-b05f-6c0ca1b46437 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.437501 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1989ec-9646-44cc-b05f-6c0ca1b46437 None admin] RBAC: Adding query filter params (role.id=1b870be20b0c4159bfbbe73593799250, scope.project.id=bd637189b43c45c28839d066df463bc3, user.id=67aea1a002a149b2aaa38b0dd4193f16) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:03.438011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1989ec-9646-44cc-b05f-6c0ca1b46437 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.442276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1989ec-9646-44cc-b05f-6c0ca1b46437 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.464080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 87/179] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/role_assignments?user.id=67aea1a002a149b2aaa38b0dd4193f16&scope.project.id=bd637189b43c45c28839d066df463bc3&role.id=1b870be20b0c4159bfbbe73593799250 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:03.464080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1989ec-9646-44cc-b05f-6c0ca1b46437 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.490460 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:03.599684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78226e09-183c-44b0-848b-79d47eb424d6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.635929 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1989ec-9646-44cc-b05f-6c0ca1b46437 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.657241 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:03.671628 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c2706-955b-4f96-bc6e-32b5b106ff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.671628 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c2706-955b-4f96-bc6e-32b5b106ff9f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.672551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c2706-955b-4f96-bc6e-32b5b106ff9f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.672551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c2706-955b-4f96-bc6e-32b5b106ff9f None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.675394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c2706-955b-4f96-bc6e-32b5b106ff9f None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.678755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c2706-955b-4f96-bc6e-32b5b106ff9f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.682377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 93/180] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:03.686153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4446e9-2586-4170-9d37-5f183fab1552 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.686443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4446e9-2586-4170-9d37-5f183fab1552 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.686720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4446e9-2586-4170-9d37-5f183fab1552 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.688349 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 94/181] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:03.689031 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4446e9-2586-4170-9d37-5f183fab1552 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.691069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99132dd-c1ee-4de0-bf18-7ba6f48bb75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.691744 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99132dd-c1ee-4de0-bf18-7ba6f48bb75f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.692031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99132dd-c1ee-4de0-bf18-7ba6f48bb75f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.692306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99132dd-c1ee-4de0-bf18-7ba6f48bb75f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.797272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99132dd-c1ee-4de0-bf18-7ba6f48bb75f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.800668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99132dd-c1ee-4de0-bf18-7ba6f48bb75f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.845709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4446e9-2586-4170-9d37-5f183fab1552 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:03.848451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:03.903164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852c2ae-0eb0-4041-b2d8-51f9c87f3047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:03.904073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852c2ae-0eb0-4041-b2d8-51f9c87f3047 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.904531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852c2ae-0eb0-4041-b2d8-51f9c87f3047 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.904992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852c2ae-0eb0-4041-b2d8-51f9c87f3047 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.906888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852c2ae-0eb0-4041-b2d8-51f9c87f3047 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:03.906888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852c2ae-0eb0-4041-b2d8-51f9c87f3047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:03.909851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852c2ae-0eb0-4041-b2d8-51f9c87f3047 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:03.924251 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 95/182] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:03.931931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ef7c9-80c0-4f4a-9dca-8bfedc96c804 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:03.932546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ef7c9-80c0-4f4a-9dca-8bfedc96c804 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:03.933060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ef7c9-80c0-4f4a-9dca-8bfedc96c804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:03.957534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 88/183] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:03.958774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99132dd-c1ee-4de0-bf18-7ba6f48bb75f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:03.964313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f82ef7c9-80c0-4f4a-9dca-8bfedc96c804 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:03.966801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f82ef7c9-80c0-4f4a-9dca-8bfedc96c804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:03.978480 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:04.086167 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99132dd-c1ee-4de0-bf18-7ba6f48bb75f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:04.095766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 96/184] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:04.097728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c8465-3a23-40c6-9192-900fcff9133f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.098025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c8465-3a23-40c6-9192-900fcff9133f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.098365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c8465-3a23-40c6-9192-900fcff9133f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.128221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3e8c8465-3a23-40c6-9192-900fcff9133f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:04.129528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3e8c8465-3a23-40c6-9192-900fcff9133f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:04.153011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae1d45-8572-4261-a883-25778150c4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:04.153891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae1d45-8572-4261-a883-25778150c4ea None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.154054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae1d45-8572-4261-a883-25778150c4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.154388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae1d45-8572-4261-a883-25778150c4ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.155081 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae1d45-8572-4261-a883-25778150c4ea None admin] RBAC: Adding query filter params (role.id=80eea410deca46e2924f27afe5136d55, scope.project.id=bd637189b43c45c28839d066df463bc3, user.id=67aea1a002a149b2aaa38b0dd4193f16) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:04.155387 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae1d45-8572-4261-a883-25778150c4ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:04.156887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae1d45-8572-4261-a883-25778150c4ea None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:04.163825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 89/185] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:03 2022] GET /identity/v3/role_assignments?user.id=67aea1a002a149b2aaa38b0dd4193f16&scope.project.id=bd637189b43c45c28839d066df463bc3&role.id=80eea410deca46e2924f27afe5136d55 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:04.164718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae1d45-8572-4261-a883-25778150c4ea None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:04.260334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 97/186] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:04.261664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c8465-3a23-40c6-9192-900fcff9133f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:04.280646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:04.305160 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:04.324470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae1d45-8572-4261-a883-25778150c4ea None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:04.395383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913f702-347d-4b9c-8e31-6ce47c058568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:04.396355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913f702-347d-4b9c-8e31-6ce47c058568 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.396682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913f702-347d-4b9c-8e31-6ce47c058568 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.396964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913f702-347d-4b9c-8e31-6ce47c058568 None admin] PATH_INFO: `/v3/users/b986a24cde6f49708cab4a161ee04e2a` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.416394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913f702-347d-4b9c-8e31-6ce47c058568 None admin] RBAC: Authorizing `identity:get_user(user_id=b986a24cde6f49708cab4a161ee04e2a)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:04.417852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c8465-3a23-40c6-9192-900fcff9133f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:04.418040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913f702-347d-4b9c-8e31-6ce47c058568 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:04.419707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 90/187] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:04 2022] GET /identity/v3/users/b986a24cde6f49708cab4a161ee04e2a => generated 283 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:04.420982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913f702-347d-4b9c-8e31-6ce47c058568 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:04.474578 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36733f35-533e-4554-9cdc-8a38a43893da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:04.475466 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36733f35-533e-4554-9cdc-8a38a43893da None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.475752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36733f35-533e-4554-9cdc-8a38a43893da None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.476029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36733f35-533e-4554-9cdc-8a38a43893da None admin] PATH_INFO: `/v3/projects/c172869d2c004a73be21e86aca1e9213` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.477453 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36733f35-533e-4554-9cdc-8a38a43893da None admin] RBAC: Authorizing `identity:get_project(project_id=c172869d2c004a73be21e86aca1e9213)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:04.478978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36733f35-533e-4554-9cdc-8a38a43893da None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:04.480889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 98/188] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:04 2022] GET /identity/v3/projects/c172869d2c004a73be21e86aca1e9213 => generated 312 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:04.482286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5b488-ad39-4883-87b5-1d46d5622362 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.482581 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5b488-ad39-4883-87b5-1d46d5622362 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.482849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5b488-ad39-4883-87b5-1d46d5622362 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.484296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 99/189] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:04.485167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5b488-ad39-4883-87b5-1d46d5622362 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:04.566685 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913f702-347d-4b9c-8e31-6ce47c058568 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:04.600083 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:04.608040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5b488-ad39-4883-87b5-1d46d5622362 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:04.616408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbddc7-9f76-45e1-929e-d499a3b2758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:04.617901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbddc7-9f76-45e1-929e-d499a3b2758b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.618204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbddc7-9f76-45e1-929e-d499a3b2758b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.618484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbddc7-9f76-45e1-929e-d499a3b2758b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.620390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbddc7-9f76-45e1-929e-d499a3b2758b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:04.622385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbddc7-9f76-45e1-929e-d499a3b2758b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:04.624411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 91/190] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:04.625073 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbddc7-9f76-45e1-929e-d499a3b2758b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:04.630111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:04.653147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d73c1-b3ba-4c48-a858-e854f8593311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:04.653982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d73c1-b3ba-4c48-a858-e854f8593311 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.654271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d73c1-b3ba-4c48-a858-e854f8593311 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.654551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d73c1-b3ba-4c48-a858-e854f8593311 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.668234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d73c1-b3ba-4c48-a858-e854f8593311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-5a0d73c1-b3ba-4c48-a858-e854f8593311 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:04.677921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:04.679367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 100/191] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:04.680153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d73c1-b3ba-4c48-a858-e854f8593311 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:04.759005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbddc7-9f76-45e1-929e-d499a3b2758b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:04.797180 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06845aa5-a24a-4861-ab58-1508cbe9051b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:04.797929 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06845aa5-a24a-4861-ab58-1508cbe9051b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.798315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06845aa5-a24a-4861-ab58-1508cbe9051b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.798739 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06845aa5-a24a-4861-ab58-1508cbe9051b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.800073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d73c1-b3ba-4c48-a858-e854f8593311 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:04.846259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750675e-a493-4e80-bdcf-5b73e2c4af6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:04.846926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750675e-a493-4e80-bdcf-5b73e2c4af6b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.847212 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750675e-a493-4e80-bdcf-5b73e2c4af6b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.847498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750675e-a493-4e80-bdcf-5b73e2c4af6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.848348 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750675e-a493-4e80-bdcf-5b73e2c4af6b None admin] RBAC: Adding query filter params (role.id=1b870be20b0c4159bfbbe73593799250, scope.project.id=c172869d2c004a73be21e86aca1e9213, user.id=67aea1a002a149b2aaa38b0dd4193f16) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:04.848697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750675e-a493-4e80-bdcf-5b73e2c4af6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:04.850155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750675e-a493-4e80-bdcf-5b73e2c4af6b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:04.856389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 101/192] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:04 2022] GET /identity/v3/role_assignments?user.id=67aea1a002a149b2aaa38b0dd4193f16&scope.project.id=c172869d2c004a73be21e86aca1e9213&role.id=1b870be20b0c4159bfbbe73593799250 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:04.857507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bf9bb-6209-4456-9fb4-8ba88f6f3b55 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:04.857797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bf9bb-6209-4456-9fb4-8ba88f6f3b55 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:04.858073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bf9bb-6209-4456-9fb4-8ba88f6f3b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:04.879182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a26bf9bb-6209-4456-9fb4-8ba88f6f3b55 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:04.880434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a26bf9bb-6209-4456-9fb4-8ba88f6f3b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:04.881806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06845aa5-a24a-4861-ab58-1508cbe9051b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:04.883405 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06845aa5-a24a-4861-ab58-1508cbe9051b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:04.887529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:04.901325 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:04.988615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 102/193] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:04.988615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bf9bb-6209-4456-9fb4-8ba88f6f3b55 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:05.054683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 92/194] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:05.055564 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06845aa5-a24a-4861-ab58-1508cbe9051b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:05.138238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bf9bb-6209-4456-9fb4-8ba88f6f3b55 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:05.184356 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06845aa5-a24a-4861-ab58-1508cbe9051b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:05.185630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdf85f-bd9e-4401-8f5f-3240e095593c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:05.187130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:05.187391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdf85f-bd9e-4401-8f5f-3240e095593c None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:05.187685 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdf85f-bd9e-4401-8f5f-3240e095593c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:05.187957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdf85f-bd9e-4401-8f5f-3240e095593c None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:05.189278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bdf85f-bd9e-4401-8f5f-3240e095593c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:05.191035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bdf85f-bd9e-4401-8f5f-3240e095593c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:05.232518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 103/195] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:04 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:05.232518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdf85f-bd9e-4401-8f5f-3240e095593c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:05.244613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1eb60-6351-4e34-8ca9-3c74a23aed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:05.244613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1eb60-6351-4e34-8ca9-3c74a23aed88 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:05.244613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1eb60-6351-4e34-8ca9-3c74a23aed88 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:05.244613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1eb60-6351-4e34-8ca9-3c74a23aed88 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:05.245031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1eb60-6351-4e34-8ca9-3c74a23aed88 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:05.245031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1eb60-6351-4e34-8ca9-3c74a23aed88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:05.245031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1eb60-6351-4e34-8ca9-3c74a23aed88 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:05.245657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:05.260586 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 93/196] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:05.262310 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1eb60-6351-4e34-8ca9-3c74a23aed88 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:05.399050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdf85f-bd9e-4401-8f5f-3240e095593c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:05.426502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1eb60-6351-4e34-8ca9-3c74a23aed88 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:05.457580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab6156-c56d-424e-bd53-a43630ba41fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:05.458943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab6156-c56d-424e-bd53-a43630ba41fc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:05.458943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab6156-c56d-424e-bd53-a43630ba41fc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:05.458943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab6156-c56d-424e-bd53-a43630ba41fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:05.472065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab6156-c56d-424e-bd53-a43630ba41fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-14ab6156-c56d-424e-bd53-a43630ba41fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:05.480291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:05.481272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:05.482065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 104/197] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:05.487838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab6156-c56d-424e-bd53-a43630ba41fc None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:05.499893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f08b84-07b3-4d6f-9205-db400671d190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:05.501658 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f08b84-07b3-4d6f-9205-db400671d190 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:05.501824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f08b84-07b3-4d6f-9205-db400671d190 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:05.502318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f08b84-07b3-4d6f-9205-db400671d190 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:05.503787 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f08b84-07b3-4d6f-9205-db400671d190 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:05.505516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f08b84-07b3-4d6f-9205-db400671d190 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:05.507309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 94/198] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:05 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:05.508202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f08b84-07b3-4d6f-9205-db400671d190 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:05.556357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:05.600942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:05.672436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab6156-c56d-424e-bd53-a43630ba41fc None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:05.692603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f08b84-07b3-4d6f-9205-db400671d190 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:05.730345 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883d513-2657-426e-a3a8-60eeb57e8fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:05.731744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883d513-2657-426e-a3a8-60eeb57e8fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:05.732034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883d513-2657-426e-a3a8-60eeb57e8fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:05.733289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883d513-2657-426e-a3a8-60eeb57e8fc9 None admin] PATH_INFO: `/v3/users/b986a24cde6f49708cab4a161ee04e2a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:05.733963 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883d513-2657-426e-a3a8-60eeb57e8fc9 None admin] RBAC: Authorizing `identity:get_user(user_id=b986a24cde6f49708cab4a161ee04e2a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:05.737763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883d513-2657-426e-a3a8-60eeb57e8fc9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:05.740259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 105/199] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:05 2022] GET /identity/v3/users/b986a24cde6f49708cab4a161ee04e2a => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:05.741114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883d513-2657-426e-a3a8-60eeb57e8fc9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:05.744999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e59408-8d89-4852-a9ab-378e99983887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:05.746322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e59408-8d89-4852-a9ab-378e99983887 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:05.746907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e59408-8d89-4852-a9ab-378e99983887 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:05.747488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e59408-8d89-4852-a9ab-378e99983887 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:05.749305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e59408-8d89-4852-a9ab-378e99983887 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:05.750036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e59408-8d89-4852-a9ab-378e99983887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:05.753200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e59408-8d89-4852-a9ab-378e99983887 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:05.769844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 95/200] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:05.771522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e59408-8d89-4852-a9ab-378e99983887 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:05.922040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883d513-2657-426e-a3a8-60eeb57e8fc9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:05.944139 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:05.961773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:05.964110 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e59408-8d89-4852-a9ab-378e99983887 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:05.970505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062cd8-fb82-41f4-a985-f28fcb7f59d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:05.972700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062cd8-fb82-41f4-a985-f28fcb7f59d1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:05.973867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062cd8-fb82-41f4-a985-f28fcb7f59d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:05.974613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93062cd8-fb82-41f4-a985-f28fcb7f59d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:05.976729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93062cd8-fb82-41f4-a985-f28fcb7f59d1 None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=8f4b4898eca3450daecc84800ff218c9, user.id=b986a24cde6f49708cab4a161ee04e2a) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:05.977435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93062cd8-fb82-41f4-a985-f28fcb7f59d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:05.980264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93062cd8-fb82-41f4-a985-f28fcb7f59d1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:05.993070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 106/201] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:05 2022] GET /identity/v3/role_assignments?user.id=b986a24cde6f49708cab4a161ee04e2a&scope.project.id=8f4b4898eca3450daecc84800ff218c9&role.id=f957e09d047b47cfa0891d25b4181263 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:05.994746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062cd8-fb82-41f4-a985-f28fcb7f59d1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:06.047098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf7093-59ee-4428-8598-ed556facf1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:06.048226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf7093-59ee-4428-8598-ed556facf1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:06.048719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf7093-59ee-4428-8598-ed556facf1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:06.049190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf7093-59ee-4428-8598-ed556facf1e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:06.209648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf7093-59ee-4428-8598-ed556facf1e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:06.212262 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf7093-59ee-4428-8598-ed556facf1e9 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:06.291404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93062cd8-fb82-41f4-a985-f28fcb7f59d1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:06.410349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:06.421421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 96/202] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:06.430000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984656b0-eee7-4d2e-af01-4ee51612b2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:06.431495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984656b0-eee7-4d2e-af01-4ee51612b2ca None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:06.431767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf7093-59ee-4428-8598-ed556facf1e9 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:06.432030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984656b0-eee7-4d2e-af01-4ee51612b2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:06.432443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984656b0-eee7-4d2e-af01-4ee51612b2ca None admin] PATH_INFO: `/v3/users/ba4f45224476433ea9ae59fb6650ccad` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:06.437996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984656b0-eee7-4d2e-af01-4ee51612b2ca None admin] RBAC: Authorizing `identity:get_user(user_id=ba4f45224476433ea9ae59fb6650ccad)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:06.441715 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984656b0-eee7-4d2e-af01-4ee51612b2ca None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:06.453109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 107/203] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:05 2022] GET /identity/v3/users/ba4f45224476433ea9ae59fb6650ccad => generated 289 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:06.459821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984656b0-eee7-4d2e-af01-4ee51612b2ca None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:06.633194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf7093-59ee-4428-8598-ed556facf1e9 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:06.691790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:06.775880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984656b0-eee7-4d2e-af01-4ee51612b2ca None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:06.777225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55f7c2-cf24-4981-b88d-ea77e38c6d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:06.778131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55f7c2-cf24-4981-b88d-ea77e38c6d9b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:06.778406 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55f7c2-cf24-4981-b88d-ea77e38c6d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:06.778692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55f7c2-cf24-4981-b88d-ea77e38c6d9b None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:06.785476 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55f7c2-cf24-4981-b88d-ea77e38c6d9b None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:06.786982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55f7c2-cf24-4981-b88d-ea77e38c6d9b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:06.789191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 97/204] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:06 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:06.798636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55f7c2-cf24-4981-b88d-ea77e38c6d9b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:06.853471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d142a18-fb05-4751-964c-f8728fa5e462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:06.854152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d142a18-fb05-4751-964c-f8728fa5e462 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:06.854437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d142a18-fb05-4751-964c-f8728fa5e462 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:06.854715 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d142a18-fb05-4751-964c-f8728fa5e462 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:06.937318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:06.994545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d142a18-fb05-4751-964c-f8728fa5e462 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:06.996006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d142a18-fb05-4751-964c-f8728fa5e462 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:07.014129 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55f7c2-cf24-4981-b88d-ea77e38c6d9b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:07.077412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b312404-54d7-4a43-9e48-0e9e35136e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:07.078098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b312404-54d7-4a43-9e48-0e9e35136e17 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:07.078382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b312404-54d7-4a43-9e48-0e9e35136e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:07.078664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b312404-54d7-4a43-9e48-0e9e35136e17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:07.080673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b312404-54d7-4a43-9e48-0e9e35136e17 None admin] RBAC: Adding query filter params (role.id=80eea410deca46e2924f27afe5136d55, scope.project.id=8f4b4898eca3450daecc84800ff218c9, user.id=b986a24cde6f49708cab4a161ee04e2a) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:07.081042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b312404-54d7-4a43-9e48-0e9e35136e17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:07.082505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b312404-54d7-4a43-9e48-0e9e35136e17 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:07.089885 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 98/205] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:06 2022] GET /identity/v3/role_assignments?user.id=b986a24cde6f49708cab4a161ee04e2a&scope.project.id=8f4b4898eca3450daecc84800ff218c9&role.id=80eea410deca46e2924f27afe5136d55 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:07.125030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:07.170762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 108/206] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:07.177023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b312404-54d7-4a43-9e48-0e9e35136e17 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:07.324980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b312404-54d7-4a43-9e48-0e9e35136e17 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:07.369491 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:07.384732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1c2cb-04e1-4a4c-be25-8dc6711c7e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:07.386444 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1c2cb-04e1-4a4c-be25-8dc6711c7e90 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:07.387055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1c2cb-04e1-4a4c-be25-8dc6711c7e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:07.387675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1c2cb-04e1-4a4c-be25-8dc6711c7e90 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:07.390108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f1c2cb-04e1-4a4c-be25-8dc6711c7e90 None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:07.393073 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f1c2cb-04e1-4a4c-be25-8dc6711c7e90 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:07.396715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 99/207] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:07 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:07.406404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d142a18-fb05-4751-964c-f8728fa5e462 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:07.600588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d142a18-fb05-4751-964c-f8728fa5e462 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:07.667121 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adbc2a-9f27-4fe0-890b-2a78a8c99367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:07.668232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adbc2a-9f27-4fe0-890b-2a78a8c99367 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:07.668712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adbc2a-9f27-4fe0-890b-2a78a8c99367 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:07.669195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adbc2a-9f27-4fe0-890b-2a78a8c99367 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:07.670867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adbc2a-9f27-4fe0-890b-2a78a8c99367 None admin] RBAC: Adding query filter params (role.id=35dbb86a4b774e0a9d862b6fa21cd947, scope.project.id=bd637189b43c45c28839d066df463bc3, user.id=ba4f45224476433ea9ae59fb6650ccad) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:07.671429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adbc2a-9f27-4fe0-890b-2a78a8c99367 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:07.674825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adbc2a-9f27-4fe0-890b-2a78a8c99367 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:07.686941 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 109/208] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:07 2022] GET /identity/v3/role_assignments?user.id=ba4f45224476433ea9ae59fb6650ccad&scope.project.id=bd637189b43c45c28839d066df463bc3&role.id=35dbb86a4b774e0a9d862b6fa21cd947 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:08.329804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2cdca-3d92-4683-8da4-2da66397e060 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.331335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2cdca-3d92-4683-8da4-2da66397e060 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.331335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2cdca-3d92-4683-8da4-2da66397e060 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.333778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 100/209] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:08.340127 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6618fb-8469-4428-b737-7362fd811ca5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.340942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6618fb-8469-4428-b737-7362fd811ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.341276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6618fb-8469-4428-b737-7362fd811ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.375131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6a6618fb-8469-4428-b737-7362fd811ca5 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:08.377770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6a6618fb-8469-4428-b737-7362fd811ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:08.381725 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc255568-bb38-4899-944f-dd4672b9a8a8 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.382201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc255568-bb38-4899-944f-dd4672b9a8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.382687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc255568-bb38-4899-944f-dd4672b9a8a8 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.388431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 101/210] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:08.396819 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacb364-6df0-490e-b63f-a41b1c8484d1 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.397209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacb364-6df0-490e-b63f-a41b1c8484d1 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.397443 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacb364-6df0-490e-b63f-a41b1c8484d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.432917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:08.441021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-1cacb364-6df0-490e-b63f-a41b1c8484d1 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:08.443345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1cacb364-6df0-490e-b63f-a41b1c8484d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:08.502886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 110/211] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:08.528747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6618fb-8469-4428-b737-7362fd811ca5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:08.559096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 102/212] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:08.585124 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacb364-6df0-490e-b63f-a41b1c8484d1 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:08.723050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6618fb-8469-4428-b737-7362fd811ca5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:08.762390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacb364-6df0-490e-b63f-a41b1c8484d1 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:08.796785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:08.810613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097914dd-0009-458c-be76-96128c9c5893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:08.811964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097914dd-0009-458c-be76-96128c9c5893 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.812444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097914dd-0009-458c-be76-96128c9c5893 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.812954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097914dd-0009-458c-be76-96128c9c5893 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.815808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097914dd-0009-458c-be76-96128c9c5893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:08.818748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097914dd-0009-458c-be76-96128c9c5893 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:08.819533 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f458a-6da2-4428-bc04-1aeb12da8bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:08.821201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f458a-6da2-4428-bc04-1aeb12da8bfb None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:08.821765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 111/213] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:08.822191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f458a-6da2-4428-bc04-1aeb12da8bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:08.822614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f458a-6da2-4428-bc04-1aeb12da8bfb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:08.828536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097914dd-0009-458c-be76-96128c9c5893 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:08.853070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f458a-6da2-4428-bc04-1aeb12da8bfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-168f458a-6da2-4428-bc04-1aeb12da8bfb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:08.866520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:08.867517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:08.868338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 103/214] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:08.874376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f458a-6da2-4428-bc04-1aeb12da8bfb None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:08.883048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:09.020125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097914dd-0009-458c-be76-96128c9c5893 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:09.041518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f458a-6da2-4428-bc04-1aeb12da8bfb None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:09.067658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad9804-3d9f-41c5-92cb-06ce473fc50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:09.068351 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad9804-3d9f-41c5-92cb-06ce473fc50a None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.068757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad9804-3d9f-41c5-92cb-06ce473fc50a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.069128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad9804-3d9f-41c5-92cb-06ce473fc50a None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.069811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad9804-3d9f-41c5-92cb-06ce473fc50a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:09.071354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad9804-3d9f-41c5-92cb-06ce473fc50a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:09.090182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971f106-5472-4662-b57c-defae697ef82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:09.091090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971f106-5472-4662-b57c-defae697ef82 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.091438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971f106-5472-4662-b57c-defae697ef82 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.091764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971f106-5472-4662-b57c-defae697ef82 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.092927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971f106-5472-4662-b57c-defae697ef82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:09.093278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971f106-5472-4662-b57c-defae697ef82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:09.095702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971f106-5472-4662-b57c-defae697ef82 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:09.112604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 112/215] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:08 2022] POST /identity/v3/users => generated 297 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:19:09.112792 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 104/216] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:09.112885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2683b5e-3fcf-4287-8909-79b0fcd790a8 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.112885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2683b5e-3fcf-4287-8909-79b0fcd790a8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.112885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2683b5e-3fcf-4287-8909-79b0fcd790a8 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.112885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 113/217] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:09.113173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971f106-5472-4662-b57c-defae697ef82 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:09.122679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8383b-5289-4d0e-8a67-06e69026cc7d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.123282 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8383b-5289-4d0e-8a67-06e69026cc7d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.123826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8383b-5289-4d0e-8a67-06e69026cc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.156198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4aa8383b-5289-4d0e-8a67-06e69026cc7d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:09.159438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4aa8383b-5289-4d0e-8a67-06e69026cc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:09.182089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:09.260659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971f106-5472-4662-b57c-defae697ef82 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:09.317416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 114/218] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:09.336296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd79ae-82c6-468f-96b1-8b96fd5e337b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:09.337917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd79ae-82c6-468f-96b1-8b96fd5e337b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.338402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd79ae-82c6-468f-96b1-8b96fd5e337b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.338710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd79ae-82c6-468f-96b1-8b96fd5e337b None admin] PATH_INFO: `/v3/users/68f50535c0234991b4672e62cfcecdae` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.341372 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8383b-5289-4d0e-8a67-06e69026cc7d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:09.361249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:09.373173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd79ae-82c6-468f-96b1-8b96fd5e337b None admin] RBAC: Authorizing `identity:get_user(user_id=68f50535c0234991b4672e62cfcecdae)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:09.379887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd79ae-82c6-468f-96b1-8b96fd5e337b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:09.391314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 105/219] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:09 2022] GET /identity/v3/users/68f50535c0234991b4672e62cfcecdae => generated 297 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:09.393937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd79ae-82c6-468f-96b1-8b96fd5e337b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:09.499102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8383b-5289-4d0e-8a67-06e69026cc7d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:09.557616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd79ae-82c6-468f-96b1-8b96fd5e337b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:09.562663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6907254-e9dd-468f-a3f9-ae4cff379997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:09.564635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6907254-e9dd-468f-a3f9-ae4cff379997 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.564635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6907254-e9dd-468f-a3f9-ae4cff379997 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.564635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6907254-e9dd-468f-a3f9-ae4cff379997 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.575045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:09.576392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6907254-e9dd-468f-a3f9-ae4cff379997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-c6907254-e9dd-468f-a3f9-ae4cff379997 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:09.586090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:09.590634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:09.592708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 115/220] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:09.594278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6907254-e9dd-468f-a3f9-ae4cff379997 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:09.640147 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437403aa-295b-4578-b6f1-162e6a63f76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:09.642878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437403aa-295b-4578-b6f1-162e6a63f76b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.643381 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437403aa-295b-4578-b6f1-162e6a63f76b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.643875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437403aa-295b-4578-b6f1-162e6a63f76b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.679343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:09.751280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6907254-e9dd-468f-a3f9-ae4cff379997 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:09.789681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437403aa-295b-4578-b6f1-162e6a63f76b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:09.792453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437403aa-295b-4578-b6f1-162e6a63f76b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:09.830050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd3848-fb8b-4399-83e0-6f1318eaf71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:09.831048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd3848-fb8b-4399-83e0-6f1318eaf71f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:09.831508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd3848-fb8b-4399-83e0-6f1318eaf71f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:09.831955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd3848-fb8b-4399-83e0-6f1318eaf71f None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:09.832966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd3848-fb8b-4399-83e0-6f1318eaf71f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:09.833454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd3848-fb8b-4399-83e0-6f1318eaf71f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:09.835195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd3848-fb8b-4399-83e0-6f1318eaf71f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:09.859233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 116/221] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:09.866235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd3848-fb8b-4399-83e0-6f1318eaf71f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:10.009403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 106/222] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:19:10.018938 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437403aa-295b-4578-b6f1-162e6a63f76b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:10.029812 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:10.034442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd3848-fb8b-4399-83e0-6f1318eaf71f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:10.074982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992057e-775a-4537-a93e-27b4caf53dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:10.077100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992057e-775a-4537-a93e-27b4caf53dee None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:10.077420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992057e-775a-4537-a93e-27b4caf53dee None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:10.077710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992057e-775a-4537-a93e-27b4caf53dee None admin] PATH_INFO: `/v3/users/b986a24cde6f49708cab4a161ee04e2a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:10.079727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992057e-775a-4537-a93e-27b4caf53dee None admin] RBAC: Authorizing `identity:get_user(user_id=b986a24cde6f49708cab4a161ee04e2a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:10.081289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992057e-775a-4537-a93e-27b4caf53dee None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:10.083454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 117/223] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:09 2022] GET /identity/v3/users/b986a24cde6f49708cab4a161ee04e2a => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:10.088305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30c0d0-b047-405c-97d6-8fde9917b101 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:10.088305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30c0d0-b047-405c-97d6-8fde9917b101 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:10.088305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30c0d0-b047-405c-97d6-8fde9917b101 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:10.092062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 118/224] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:10.093892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92b99d-12f5-45ab-b045-38064a4f4806 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:10.094187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92b99d-12f5-45ab-b045-38064a4f4806 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:10.094512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92b99d-12f5-45ab-b045-38064a4f4806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:10.117655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8b92b99d-12f5-45ab-b045-38064a4f4806 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:10.119577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8b92b99d-12f5-45ab-b045-38064a4f4806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:10.124858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:10.175303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437403aa-295b-4578-b6f1-162e6a63f76b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:10.214538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 119/225] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:10.216696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92b99d-12f5-45ab-b045-38064a4f4806 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:10.241655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cf29d-bd6a-4ded-8dd8-17e0d2564cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:10.242315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cf29d-bd6a-4ded-8dd8-17e0d2564cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:10.242613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cf29d-bd6a-4ded-8dd8-17e0d2564cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:10.242895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cf29d-bd6a-4ded-8dd8-17e0d2564cb6 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:10.244371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cf29d-bd6a-4ded-8dd8-17e0d2564cb6 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:10.246003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cf29d-bd6a-4ded-8dd8-17e0d2564cb6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:10.248441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 107/226] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:10 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:10.249481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cf29d-bd6a-4ded-8dd8-17e0d2564cb6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:10.346663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:10.381163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cf29d-bd6a-4ded-8dd8-17e0d2564cb6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:10.421868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92b99d-12f5-45ab-b045-38064a4f4806 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:10.436257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a526d-6059-49ec-a796-fbbda9e4b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:10.437580 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a526d-6059-49ec-a796-fbbda9e4b283 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:10.438666 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a526d-6059-49ec-a796-fbbda9e4b283 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:10.439348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a526d-6059-49ec-a796-fbbda9e4b283 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:10.453104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a526d-6059-49ec-a796-fbbda9e4b283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-fe0a526d-6059-49ec-a796-fbbda9e4b283 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:10.462758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:10.463904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:10.464959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:10.464959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:10.464959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:10.464959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:10.464959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:10.464959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:10.464959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:10.464959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:10.464959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:10.464959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:10.464959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:10.464959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:10.467361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 108/227] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:10.468456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a526d-6059-49ec-a796-fbbda9e4b283 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:10.497148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:10.510949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2db2d8-19ed-483a-b75c-9ba8dd787fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:10.513530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2db2d8-19ed-483a-b75c-9ba8dd787fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:10.514068 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2db2d8-19ed-483a-b75c-9ba8dd787fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:10.514570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2db2d8-19ed-483a-b75c-9ba8dd787fa8 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:10.519607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2db2d8-19ed-483a-b75c-9ba8dd787fa8 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:10.522246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2db2d8-19ed-483a-b75c-9ba8dd787fa8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:10.532434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 120/228] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:10 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:10.532434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2db2d8-19ed-483a-b75c-9ba8dd787fa8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:10.631206 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a526d-6059-49ec-a796-fbbda9e4b283 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:10.676713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4299-929e-4cd6-8359-64018a11a14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:10.677410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4299-929e-4cd6-8359-64018a11a14d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:10.677701 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4299-929e-4cd6-8359-64018a11a14d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:10.677985 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4299-929e-4cd6-8359-64018a11a14d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:10.678885 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd4299-929e-4cd6-8359-64018a11a14d None admin] RBAC: Adding query filter params (role.id=1b870be20b0c4159bfbbe73593799250, scope.project.id=8f4b4898eca3450daecc84800ff218c9, user.id=68f50535c0234991b4672e62cfcecdae) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:10.679211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd4299-929e-4cd6-8359-64018a11a14d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:10.680764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd4299-929e-4cd6-8359-64018a11a14d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:10.682944 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:10.689827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 109/229] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:10 2022] GET /identity/v3/role_assignments?user.id=68f50535c0234991b4672e62cfcecdae&scope.project.id=8f4b4898eca3450daecc84800ff218c9&role.id=1b870be20b0c4159bfbbe73593799250 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:10.691000 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4299-929e-4cd6-8359-64018a11a14d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:10.745587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2db2d8-19ed-483a-b75c-9ba8dd787fa8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:10.818938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09810884-65d4-4d0f-b75f-ddc72b1f08c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:10.820250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09810884-65d4-4d0f-b75f-ddc72b1f08c1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:10.821547 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09810884-65d4-4d0f-b75f-ddc72b1f08c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:10.822179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09810884-65d4-4d0f-b75f-ddc72b1f08c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:10.824354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09810884-65d4-4d0f-b75f-ddc72b1f08c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:10.825142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09810884-65d4-4d0f-b75f-ddc72b1f08c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:10.827758 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09810884-65d4-4d0f-b75f-ddc72b1f08c1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:10.847034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 121/230] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:10.858279 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09810884-65d4-4d0f-b75f-ddc72b1f08c1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:10.919758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4299-929e-4cd6-8359-64018a11a14d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:10.982383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:11.001077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfe3b1-5c86-4ec0-90ee-fa7ee45998e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:11.002475 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfe3b1-5c86-4ec0-90ee-fa7ee45998e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:11.002763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfe3b1-5c86-4ec0-90ee-fa7ee45998e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:11.003044 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfe3b1-5c86-4ec0-90ee-fa7ee45998e6 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9/users/b986a24cde6f49708cab4a161ee04e2a/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:11.015130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfe3b1-5c86-4ec0-90ee-fa7ee45998e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f4b4898eca3450daecc84800ff218c9, user_id=b986a24cde6f49708cab4a161ee04e2a, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:11.018103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfe3b1-5c86-4ec0-90ee-fa7ee45998e6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:11.043808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09810884-65d4-4d0f-b75f-ddc72b1f08c1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:11.076814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 110/231] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:19:10 2022] PUT /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9/users/b986a24cde6f49708cab4a161ee04e2a/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:11.087127 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364251cd-e5a3-4144-9a81-166898b7cf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:11.088022 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364251cd-e5a3-4144-9a81-166898b7cf03 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:11.088570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364251cd-e5a3-4144-9a81-166898b7cf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:11.088861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364251cd-e5a3-4144-9a81-166898b7cf03 None admin] PATH_INFO: `/v3/users/b986a24cde6f49708cab4a161ee04e2a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:11.090421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364251cd-e5a3-4144-9a81-166898b7cf03 None admin] RBAC: Authorizing `identity:get_user(user_id=b986a24cde6f49708cab4a161ee04e2a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:11.092425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364251cd-e5a3-4144-9a81-166898b7cf03 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:11.095761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 122/232] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:10 2022] GET /identity/v3/users/b986a24cde6f49708cab4a161ee04e2a => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:11.103728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfe3b1-5c86-4ec0-90ee-fa7ee45998e6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:11.241609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:11.305991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfe3b1-5c86-4ec0-90ee-fa7ee45998e6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:11.356259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cb59af-1c8e-4d2e-8c41-582cd16d26e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:11.358053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb59af-1c8e-4d2e-8c41-582cd16d26e8 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:11.358605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb59af-1c8e-4d2e-8c41-582cd16d26e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:11.359092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cb59af-1c8e-4d2e-8c41-582cd16d26e8 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:11.361883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb59af-1c8e-4d2e-8c41-582cd16d26e8 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:11.363750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cb59af-1c8e-4d2e-8c41-582cd16d26e8 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:11.366385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 111/233] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:11 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:11.374605 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364251cd-e5a3-4144-9a81-166898b7cf03 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:11.501840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364251cd-e5a3-4144-9a81-166898b7cf03 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:11.534342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:11.541090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c2c36-cc95-4963-b8e3-e20a729509c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:11.542034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2c36-cc95-4963-b8e3-e20a729509c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:11.542319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2c36-cc95-4963-b8e3-e20a729509c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:11.542597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c2c36-cc95-4963-b8e3-e20a729509c6 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9/users/b986a24cde6f49708cab4a161ee04e2a/roles/80eea410deca46e2924f27afe5136d55` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:11.545924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c2c36-cc95-4963-b8e3-e20a729509c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f4b4898eca3450daecc84800ff218c9, user_id=b986a24cde6f49708cab4a161ee04e2a, role_id=80eea410deca46e2924f27afe5136d55)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:11.548369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c2c36-cc95-4963-b8e3-e20a729509c6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:11.565553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 123/234] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:19:11 2022] PUT /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9/users/b986a24cde6f49708cab4a161ee04e2a/roles/80eea410deca46e2924f27afe5136d55 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:11.726730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e8b1b-3792-4327-a6b5-9d068c442936 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:11.727263 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e8b1b-3792-4327-a6b5-9d068c442936 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:11.727779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e8b1b-3792-4327-a6b5-9d068c442936 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:11.731243 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 112/235] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:11.737785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f5901-c10b-4d59-9bec-ee38588b31b9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:11.738545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f5901-c10b-4d59-9bec-ee38588b31b9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:11.739028 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f5901-c10b-4d59-9bec-ee38588b31b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:11.765823 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5f9f5901-c10b-4d59-9bec-ee38588b31b9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:11.768020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5f9f5901-c10b-4d59-9bec-ee38588b31b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:11.858150 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 124/236] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:11.883373 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e8b1b-3792-4327-a6b5-9d068c442936 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:12.005202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e8b1b-3792-4327-a6b5-9d068c442936 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:12.041767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:12.052405 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a9a23-0622-4a54-8fc7-b5e749435fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:12.053260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a9a23-0622-4a54-8fc7-b5e749435fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:12.053548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a9a23-0622-4a54-8fc7-b5e749435fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:12.053831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a9a23-0622-4a54-8fc7-b5e749435fa4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:12.075913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a9a23-0622-4a54-8fc7-b5e749435fa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-9b2a9a23-0622-4a54-8fc7-b5e749435fa4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:12.084817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:12.085761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:12.086536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 113/237] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:12.093352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f5901-c10b-4d59-9bec-ee38588b31b9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:12.237442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f5901-c10b-4d59-9bec-ee38588b31b9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:12.273171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:12.296762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181cdce0-e1d8-48b3-89dd-eec0931d5c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:12.297590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181cdce0-e1d8-48b3-89dd-eec0931d5c21 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:12.297756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181cdce0-e1d8-48b3-89dd-eec0931d5c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:12.298019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181cdce0-e1d8-48b3-89dd-eec0931d5c21 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:12.298998 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181cdce0-e1d8-48b3-89dd-eec0931d5c21 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:12.299308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181cdce0-e1d8-48b3-89dd-eec0931d5c21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:12.300850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181cdce0-e1d8-48b3-89dd-eec0931d5c21 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:12.312336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 125/238] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:12.318643 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a9a23-0622-4a54-8fc7-b5e749435fa4 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:12.451702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a9a23-0622-4a54-8fc7-b5e749435fa4 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:12.506740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadb614-ad5e-4f5a-9de8-f1b4b9d4b555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:12.507655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadb614-ad5e-4f5a-9de8-f1b4b9d4b555 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:12.507945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadb614-ad5e-4f5a-9de8-f1b4b9d4b555 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:12.508229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadb614-ad5e-4f5a-9de8-f1b4b9d4b555 None admin] PATH_INFO: `/v3/users/d150ec2538804dc788b00129edbbcd44` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:12.530809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadb614-ad5e-4f5a-9de8-f1b4b9d4b555 None admin] RBAC: Authorizing `identity:get_user(user_id=d150ec2538804dc788b00129edbbcd44)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:12.532866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadb614-ad5e-4f5a-9de8-f1b4b9d4b555 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:12.534976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 114/239] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:12 2022] GET /identity/v3/users/d150ec2538804dc788b00129edbbcd44 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:12.541008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181cdce0-e1d8-48b3-89dd-eec0931d5c21 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:12.714756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181cdce0-e1d8-48b3-89dd-eec0931d5c21 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:12.770263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6eec2-1898-484a-ab1f-88d1ccf90706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:12.770953 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6eec2-1898-484a-ab1f-88d1ccf90706 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:12.771244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6eec2-1898-484a-ab1f-88d1ccf90706 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:12.771528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6eec2-1898-484a-ab1f-88d1ccf90706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:12.786935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a27ff-0f1b-4c4e-8be3-98433bbbe828 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:12.787354 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a27ff-0f1b-4c4e-8be3-98433bbbe828 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:12.787529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a27ff-0f1b-4c4e-8be3-98433bbbe828 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:12.789161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 115/240] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:12.794825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0fd45-44dc-453f-aa90-400ece88adab None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:12.795193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0fd45-44dc-453f-aa90-400ece88adab None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:12.795515 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0fd45-44dc-453f-aa90-400ece88adab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:12.821832 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-a2e0fd45-44dc-453f-aa90-400ece88adab None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:12.823047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a2e0fd45-44dc-453f-aa90-400ece88adab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:12.876348 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:12.957278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6eec2-1898-484a-ab1f-88d1ccf90706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:12.960768 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6eec2-1898-484a-ab1f-88d1ccf90706 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:12.965067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:13.034874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 116/241] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:13.072707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0fd45-44dc-453f-aa90-400ece88adab None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:13.194156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 126/242] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:13.201890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6eec2-1898-484a-ab1f-88d1ccf90706 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:13.284906 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0fd45-44dc-453f-aa90-400ece88adab None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:13.344048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef0743-ba58-432f-894a-6995d21a6292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:13.345691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef0743-ba58-432f-894a-6995d21a6292 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.345977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef0743-ba58-432f-894a-6995d21a6292 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.346260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef0743-ba58-432f-894a-6995d21a6292 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.347632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef0743-ba58-432f-894a-6995d21a6292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:13.349168 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef0743-ba58-432f-894a-6995d21a6292 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:13.351097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 117/243] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:13.356650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef0743-ba58-432f-894a-6995d21a6292 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:13.384706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:13.426783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6eec2-1898-484a-ab1f-88d1ccf90706 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:13.446877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:13.497016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b6350-6400-498d-8682-7154a23b4333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:13.498508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b6350-6400-498d-8682-7154a23b4333 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.499019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b6350-6400-498d-8682-7154a23b4333 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.499517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b6350-6400-498d-8682-7154a23b4333 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.501872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b6350-6400-498d-8682-7154a23b4333 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:13.504696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b6350-6400-498d-8682-7154a23b4333 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:13.507948 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 127/244] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:13 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:13.510355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a8814-a8fb-41d4-b778-0e0032295896 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.510851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a8814-a8fb-41d4-b778-0e0032295896 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.511548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a8814-a8fb-41d4-b778-0e0032295896 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.514749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 128/245] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:13.516977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a8814-a8fb-41d4-b778-0e0032295896 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:13.534072 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef0743-ba58-432f-894a-6995d21a6292 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:13.574684 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04711699-b48d-45c3-8245-bb7f00fcb07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:13.576208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04711699-b48d-45c3-8245-bb7f00fcb07f None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.576778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04711699-b48d-45c3-8245-bb7f00fcb07f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.577194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04711699-b48d-45c3-8245-bb7f00fcb07f None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.578081 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04711699-b48d-45c3-8245-bb7f00fcb07f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:13.580232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04711699-b48d-45c3-8245-bb7f00fcb07f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:13.612984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 118/246] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:13 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:19:13.614763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da970a9-a689-49f7-b4b6-534657876269 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.615312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da970a9-a689-49f7-b4b6-534657876269 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.615734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da970a9-a689-49f7-b4b6-534657876269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.640144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7da970a9-a689-49f7-b4b6-534657876269 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:13.642157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7da970a9-a689-49f7-b4b6-534657876269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:13.716829 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a8814-a8fb-41d4-b778-0e0032295896 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:13.738491 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 119/247] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:13.776123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da970a9-a689-49f7-b4b6-534657876269 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:13.800520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49eac0e-58ae-4f14-be40-71bddbbda9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:13.800520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49eac0e-58ae-4f14-be40-71bddbbda9af None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:13.800520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49eac0e-58ae-4f14-be40-71bddbbda9af None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:13.800520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49eac0e-58ae-4f14-be40-71bddbbda9af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:13.801002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49eac0e-58ae-4f14-be40-71bddbbda9af None admin] RBAC: Adding query filter params (role.id=35dbb86a4b774e0a9d862b6fa21cd947, scope.project.id=8f4b4898eca3450daecc84800ff218c9, user.id=d150ec2538804dc788b00129edbbcd44) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:13.801002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49eac0e-58ae-4f14-be40-71bddbbda9af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:13.801002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49eac0e-58ae-4f14-be40-71bddbbda9af None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:13.816514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 129/248] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:13 2022] GET /identity/v3/role_assignments?user.id=d150ec2538804dc788b00129edbbcd44&scope.project.id=8f4b4898eca3450daecc84800ff218c9&role.id=35dbb86a4b774e0a9d862b6fa21cd947 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:13.850675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:13.998682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da970a9-a689-49f7-b4b6-534657876269 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:14.039577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ab0f2-b311-4a50-9581-0a64c3082e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:14.040401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ab0f2-b311-4a50-9581-0a64c3082e4e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.040727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ab0f2-b311-4a50-9581-0a64c3082e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.041008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ab0f2-b311-4a50-9581-0a64c3082e4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.053672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ab0f2-b311-4a50-9581-0a64c3082e4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-2d3ab0f2-b311-4a50-9581-0a64c3082e4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:14.061815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:14.063065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:14.063978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 120/249] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:14.070639 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49eac0e-58ae-4f14-be40-71bddbbda9af None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:14.090229 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:14.210723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49eac0e-58ae-4f14-be40-71bddbbda9af None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:14.257521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54c786-9cbd-45ea-b641-f9ac8e158687 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.257861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54c786-9cbd-45ea-b641-f9ac8e158687 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.258086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54c786-9cbd-45ea-b641-f9ac8e158687 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.259695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 121/250] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:19:14.263394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f46df-683a-4d29-9046-72e12038ac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:14.264166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f46df-683a-4d29-9046-72e12038ac88 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.264452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f46df-683a-4d29-9046-72e12038ac88 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.264801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f46df-683a-4d29-9046-72e12038ac88 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.265518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8125c-9c77-42be-9f1b-6db34f6eb07e None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.265695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f46df-683a-4d29-9046-72e12038ac88 None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:14.265871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8125c-9c77-42be-9f1b-6db34f6eb07e None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.266038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f46df-683a-4d29-9046-72e12038ac88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:14.266188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8125c-9c77-42be-9f1b-6db34f6eb07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.267316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f46df-683a-4d29-9046-72e12038ac88 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:14.285239 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 130/251] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:14.291274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-dda8125c-9c77-42be-9f1b-6db34f6eb07e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:14.291566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f46df-683a-4d29-9046-72e12038ac88 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:14.292692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-dda8125c-9c77-42be-9f1b-6db34f6eb07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:14.408061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 122/252] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:14.426869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:14.436294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8125c-9c77-42be-9f1b-6db34f6eb07e None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:14.464578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:14.466624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f46df-683a-4d29-9046-72e12038ac88 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:14.515210 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebb1cf-8386-476e-895e-c08a0ada1836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:14.517046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebb1cf-8386-476e-895e-c08a0ada1836 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.517528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebb1cf-8386-476e-895e-c08a0ada1836 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.518252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebb1cf-8386-476e-895e-c08a0ada1836 None admin] PATH_INFO: `/v3/users/68f50535c0234991b4672e62cfcecdae` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.520090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ebb1cf-8386-476e-895e-c08a0ada1836 None admin] RBAC: Authorizing `identity:get_user(user_id=68f50535c0234991b4672e62cfcecdae)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:14.522684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ebb1cf-8386-476e-895e-c08a0ada1836 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:14.525783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 131/253] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:14 2022] GET /identity/v3/users/68f50535c0234991b4672e62cfcecdae => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:14.533126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebb1cf-8386-476e-895e-c08a0ada1836 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:14.584424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8125c-9c77-42be-9f1b-6db34f6eb07e None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:14.645868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f0faf-c4b7-4e3f-a0de-a6592495b8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:14.646723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f0faf-c4b7-4e3f-a0de-a6592495b8cf None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.646972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f0faf-c4b7-4e3f-a0de-a6592495b8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.647250 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f0faf-c4b7-4e3f-a0de-a6592495b8cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.684776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f0faf-c4b7-4e3f-a0de-a6592495b8cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:14.696691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebb1cf-8386-476e-895e-c08a0ada1836 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-9e8f0faf-c4b7-4e3f-a0de-a6592495b8cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:14.703157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:14.704943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:14.706483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:14.706483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:14.706483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:14.706483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:14.706483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:14.706483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:14.706483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:14.706483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:14.706483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:14.706483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:14.706483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:14.706483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:14.712816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 123/254] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:14.716581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2b115-2f46-455b-982b-bdbebca0c310 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.717518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2b115-2f46-455b-982b-bdbebca0c310 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.718280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2b115-2f46-455b-982b-bdbebca0c310 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.722610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 124/255] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:14.724623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2b115-2f46-455b-982b-bdbebca0c310 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:14.812109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423d389-d644-43b5-a992-df9b17633944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:14.814588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423d389-d644-43b5-a992-df9b17633944 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.815565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423d389-d644-43b5-a992-df9b17633944 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.817691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423d389-d644-43b5-a992-df9b17633944 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.825315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423d389-d644-43b5-a992-df9b17633944 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:14.827122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423d389-d644-43b5-a992-df9b17633944 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:14.837969 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 132/256] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:14 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:14.839774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abd4c3-33ce-4428-9668-f517c8eb0ed6 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:14.840270 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abd4c3-33ce-4428-9668-f517c8eb0ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:14.842869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abd4c3-33ce-4428-9668-f517c8eb0ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:14.879581 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-26abd4c3-33ce-4428-9668-f517c8eb0ed6 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:14.883300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-26abd4c3-33ce-4428-9668-f517c8eb0ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:14.900401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:14.908913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:14.962877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2b115-2f46-455b-982b-bdbebca0c310 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.037958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b66600-67c6-4d2e-8f40-3d0e4a7ae7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.038654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b66600-67c6-4d2e-8f40-3d0e4a7ae7ce None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.038943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b66600-67c6-4d2e-8f40-3d0e4a7ae7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.039225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b66600-67c6-4d2e-8f40-3d0e4a7ae7ce None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.040005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b66600-67c6-4d2e-8f40-3d0e4a7ae7ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:15.040318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b66600-67c6-4d2e-8f40-3d0e4a7ae7ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:15.041783 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b66600-67c6-4d2e-8f40-3d0e4a7ae7ce None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:15.045460 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 133/257] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:15.046225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abd4c3-33ce-4428-9668-f517c8eb0ed6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:15.054673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 125/258] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:15.061663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b66600-67c6-4d2e-8f40-3d0e4a7ae7ce None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:15.192042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b66600-67c6-4d2e-8f40-3d0e4a7ae7ce None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.200786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abd4c3-33ce-4428-9668-f517c8eb0ed6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.243105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b9063-1c48-4ae5-bfa5-2e883a021386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.244797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b9063-1c48-4ae5-bfa5-2e883a021386 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.245833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b9063-1c48-4ae5-bfa5-2e883a021386 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.246776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b9063-1c48-4ae5-bfa5-2e883a021386 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9/users/68f50535c0234991b4672e62cfcecdae/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.250200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b9063-1c48-4ae5-bfa5-2e883a021386 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f4b4898eca3450daecc84800ff218c9, user_id=68f50535c0234991b4672e62cfcecdae, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:15.251045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7be1e-ebb2-4cfb-8abe-90b253f98b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.252161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7be1e-ebb2-4cfb-8abe-90b253f98b37 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.252690 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7be1e-ebb2-4cfb-8abe-90b253f98b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.254426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b9063-1c48-4ae5-bfa5-2e883a021386 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:15.254957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7be1e-ebb2-4cfb-8abe-90b253f98b37 None admin] PATH_INFO: `/v3/users/b986a24cde6f49708cab4a161ee04e2a` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.258851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7be1e-ebb2-4cfb-8abe-90b253f98b37 None admin] RBAC: Authorizing `identity:get_user(user_id=b986a24cde6f49708cab4a161ee04e2a)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:15.260662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7be1e-ebb2-4cfb-8abe-90b253f98b37 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:15.262613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 126/259] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:15 2022] GET /identity/v3/users/b986a24cde6f49708cab4a161ee04e2a => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:15.263908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7be1e-ebb2-4cfb-8abe-90b253f98b37 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:15.291477 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:15.294897 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 134/260] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:19:15 2022] PUT /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9/users/68f50535c0234991b4672e62cfcecdae/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:15.295793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b9063-1c48-4ae5-bfa5-2e883a021386 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:15.302762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:15.563312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b9063-1c48-4ae5-bfa5-2e883a021386 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.579417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7be1e-ebb2-4cfb-8abe-90b253f98b37 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.619996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f3662-de0d-40fa-91b5-1c0b896792ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.620808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f3662-de0d-40fa-91b5-1c0b896792ba None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.621096 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f3662-de0d-40fa-91b5-1c0b896792ba None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.621373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f3662-de0d-40fa-91b5-1c0b896792ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.674455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af2738-e16d-4e3a-9a9c-01a47d4d6bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.675308 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af2738-e16d-4e3a-9a9c-01a47d4d6bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.675601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af2738-e16d-4e3a-9a9c-01a47d4d6bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.675878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af2738-e16d-4e3a-9a9c-01a47d4d6bb3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.693481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af2738-e16d-4e3a-9a9c-01a47d4d6bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-67af2738-e16d-4e3a-9a9c-01a47d4d6bb3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:15.701951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:15.702936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:15.704446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 127/261] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:15.710214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af2738-e16d-4e3a-9a9c-01a47d4d6bb3 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:15.717564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f3662-de0d-40fa-91b5-1c0b896792ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:15.720052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f3662-de0d-40fa-91b5-1c0b896792ba None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:15.726146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:15.824792 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:15.898765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af2738-e16d-4e3a-9a9c-01a47d4d6bb3 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:15.967116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 135/262] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:15.970213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f3662-de0d-40fa-91b5-1c0b896792ba None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:15.992143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8dea3f-70bb-4674-8d79-bccb008bed5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:15.992995 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8dea3f-70bb-4674-8d79-bccb008bed5d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:15.993433 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8dea3f-70bb-4674-8d79-bccb008bed5d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:15.993859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8dea3f-70bb-4674-8d79-bccb008bed5d None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:15.994645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8dea3f-70bb-4674-8d79-bccb008bed5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:15.995045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8dea3f-70bb-4674-8d79-bccb008bed5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:15.997251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8dea3f-70bb-4674-8d79-bccb008bed5d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:16.013787 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 128/263] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:16.019633 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8dea3f-70bb-4674-8d79-bccb008bed5d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.204292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8dea3f-70bb-4674-8d79-bccb008bed5d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:16.251249 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:16.261832 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d462dcb-3c66-4ecd-bf44-8817d8816d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:16.262746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d462dcb-3c66-4ecd-bf44-8817d8816d19 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:16.263032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d462dcb-3c66-4ecd-bf44-8817d8816d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:16.263325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d462dcb-3c66-4ecd-bf44-8817d8816d19 None admin] PATH_INFO: `/v3/users/b986a24cde6f49708cab4a161ee04e2a` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:16.264499 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f3662-de0d-40fa-91b5-1c0b896792ba None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:16.264696 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d462dcb-3c66-4ecd-bf44-8817d8816d19 None admin] RBAC: Authorizing `identity:get_user(user_id=b986a24cde6f49708cab4a161ee04e2a)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:16.266089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d462dcb-3c66-4ecd-bf44-8817d8816d19 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:16.266642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:16.274591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 129/264] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/users/b986a24cde6f49708cab4a161ee04e2a => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:16.275769 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d462dcb-3c66-4ecd-bf44-8817d8816d19 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.327778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de40159-36ef-4d66-8070-5b6441f544fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:16.329396 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de40159-36ef-4d66-8070-5b6441f544fd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:16.329692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de40159-36ef-4d66-8070-5b6441f544fd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:16.329950 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de40159-36ef-4d66-8070-5b6441f544fd None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:16.333840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de40159-36ef-4d66-8070-5b6441f544fd None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:16.340752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de40159-36ef-4d66-8070-5b6441f544fd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:16.344012 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 136/265] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:15 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:16.363719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de40159-36ef-4d66-8070-5b6441f544fd None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.446151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d462dcb-3c66-4ecd-bf44-8817d8816d19 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:16.510342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5936a-15a2-45c5-9292-e6742429d6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:16.510948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5936a-15a2-45c5-9292-e6742429d6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:16.511236 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5936a-15a2-45c5-9292-e6742429d6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:16.511512 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5936a-15a2-45c5-9292-e6742429d6e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:16.630015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5936a-15a2-45c5-9292-e6742429d6e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:16.631504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5936a-15a2-45c5-9292-e6742429d6e6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:16.635682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:16.703757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de40159-36ef-4d66-8070-5b6441f544fd None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:16.805305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca35a5-50dd-4581-97c8-561e8461e6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:16.805986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca35a5-50dd-4581-97c8-561e8461e6ca None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:16.806270 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca35a5-50dd-4581-97c8-561e8461e6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:16.806543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca35a5-50dd-4581-97c8-561e8461e6ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:16.807397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca35a5-50dd-4581-97c8-561e8461e6ca None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=8f4b4898eca3450daecc84800ff218c9, user.id=b986a24cde6f49708cab4a161ee04e2a) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:16.807709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca35a5-50dd-4581-97c8-561e8461e6ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:16.809292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca35a5-50dd-4581-97c8-561e8461e6ca None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:16.823672 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 137/266] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/role_assignments?user.id=b986a24cde6f49708cab4a161ee04e2a&scope.project.id=8f4b4898eca3450daecc84800ff218c9&role.id=f957e09d047b47cfa0891d25b4181263 => generated 608 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:16.878264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 130/267] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:19:16.884116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca35a5-50dd-4581-97c8-561e8461e6ca None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:16.946849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:17.042722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca35a5-50dd-4581-97c8-561e8461e6ca None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:17.098600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8a707-f66a-4d91-afd8-c9f3ee503c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:17.099840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8a707-f66a-4d91-afd8-c9f3ee503c6d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.100328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8a707-f66a-4d91-afd8-c9f3ee503c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.100836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8a707-f66a-4d91-afd8-c9f3ee503c6d None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.102360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8a707-f66a-4d91-afd8-c9f3ee503c6d None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:17.104068 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8a707-f66a-4d91-afd8-c9f3ee503c6d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:17.106541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 138/268] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:16 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:17.112648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5936a-15a2-45c5-9292-e6742429d6e6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:17.236246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab755645-47b5-4ed4-95e8-8d773271f06b None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.237182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab755645-47b5-4ed4-95e8-8d773271f06b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.237830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab755645-47b5-4ed4-95e8-8d773271f06b None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.238444 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:17.242240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 139/269] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:17.248007 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5936a-15a2-45c5-9292-e6742429d6e6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:17.251432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac682ee-dcf6-4cc0-9301-2748a5a8e757 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.252205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac682ee-dcf6-4cc0-9301-2748a5a8e757 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.255131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac682ee-dcf6-4cc0-9301-2748a5a8e757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.309404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-bac682ee-dcf6-4cc0-9301-2748a5a8e757 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:17.313352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bac682ee-dcf6-4cc0-9301-2748a5a8e757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:17.344538 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70159a-3cb6-4e5d-9ab2-f50067bc58e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:17.346098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70159a-3cb6-4e5d-9ab2-f50067bc58e7 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.346888 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70159a-3cb6-4e5d-9ab2-f50067bc58e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.347668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70159a-3cb6-4e5d-9ab2-f50067bc58e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.349772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70159a-3cb6-4e5d-9ab2-f50067bc58e7 None admin] RBAC: Adding query filter params (role.id=80eea410deca46e2924f27afe5136d55, scope.project.id=8f4b4898eca3450daecc84800ff218c9, user.id=b986a24cde6f49708cab4a161ee04e2a) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:17.354458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70159a-3cb6-4e5d-9ab2-f50067bc58e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:17.357613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70159a-3cb6-4e5d-9ab2-f50067bc58e7 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:17.380640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 131/270] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:17 2022] GET /identity/v3/role_assignments?user.id=b986a24cde6f49708cab4a161ee04e2a&scope.project.id=8f4b4898eca3450daecc84800ff218c9&role.id=80eea410deca46e2924f27afe5136d55 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:17.442882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 140/271] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:17.511305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70159a-3cb6-4e5d-9ab2-f50067bc58e7 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:17.554762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5fe00-be5c-4aef-af2f-03d9c27c08c3 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.555865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5fe00-be5c-4aef-af2f-03d9c27c08c3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.556636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5fe00-be5c-4aef-af2f-03d9c27c08c3 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.560339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 141/272] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:19:17.575004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a343a7-20c3-48b5-ad25-bd467f3d0332 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.575726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a343a7-20c3-48b5-ad25-bd467f3d0332 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.576352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a343a7-20c3-48b5-ad25-bd467f3d0332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.605863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-32a343a7-20c3-48b5-ad25-bd467f3d0332 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:17.606993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-32a343a7-20c3-48b5-ad25-bd467f3d0332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:17.642691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:17.654630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70159a-3cb6-4e5d-9ab2-f50067bc58e7 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:17.701432 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cadc4-1849-49fe-a19b-07bc88528021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:17.702277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cadc4-1849-49fe-a19b-07bc88528021 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.702743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cadc4-1849-49fe-a19b-07bc88528021 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.703556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cadc4-1849-49fe-a19b-07bc88528021 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.708858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 142/273] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:17.710802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14260421-e346-49dd-a694-cb8253b9cb29 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.711299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14260421-e346-49dd-a694-cb8253b9cb29 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.711727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14260421-e346-49dd-a694-cb8253b9cb29 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.713357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 143/274] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:17.718222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55cadc4-1849-49fe-a19b-07bc88528021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:17.720808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b497733-8bcd-45a5-96a5-f6db499ffaa8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.721306 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b497733-8bcd-45a5-96a5-f6db499ffaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.721941 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b497733-8bcd-45a5-96a5-f6db499ffaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-f55cadc4-1849-49fe-a19b-07bc88528021 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:17.728010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:17.729291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:17.730336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:17.730336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:17.730336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:17.730336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:17.730336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:17.730336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:17.730336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:17.730336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:17.730336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:17.730336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:17.730336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:17.730336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:17.733362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 132/275] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:17.739748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cadc4-1849-49fe-a19b-07bc88528021 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:17.749535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9b497733-8bcd-45a5-96a5-f6db499ffaa8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:17.752687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9b497733-8bcd-45a5-96a5-f6db499ffaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:17.783718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:17.853122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 144/276] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:17.854702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b497733-8bcd-45a5-96a5-f6db499ffaa8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:17.898350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cadc4-1849-49fe-a19b-07bc88528021 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:17.939863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c09f98-6f35-49d1-bd73-3b63bd9280db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:17.940581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c09f98-6f35-49d1-bd73-3b63bd9280db None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:17.940892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c09f98-6f35-49d1-bd73-3b63bd9280db None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:17.941153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c09f98-6f35-49d1-bd73-3b63bd9280db None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:17.941898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c09f98-6f35-49d1-bd73-3b63bd9280db None admin] RBAC: Adding query filter params (name=reader) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:17.942214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c09f98-6f35-49d1-bd73-3b63bd9280db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:17.943635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c09f98-6f35-49d1-bd73-3b63bd9280db None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:17.955598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 133/277] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:17.956534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c09f98-6f35-49d1-bd73-3b63bd9280db None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:17.991945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:18.006346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b497733-8bcd-45a5-96a5-f6db499ffaa8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.050897 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e428a9-2d99-41c3-8e15-45d63038d379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.052281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e428a9-2d99-41c3-8e15-45d63038d379 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.052812 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e428a9-2d99-41c3-8e15-45d63038d379 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.053372 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e428a9-2d99-41c3-8e15-45d63038d379 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.055061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e428a9-2d99-41c3-8e15-45d63038d379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.056750 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e428a9-2d99-41c3-8e15-45d63038d379 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.063931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 145/278] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:18.063931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e428a9-2d99-41c3-8e15-45d63038d379 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.077925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:18.098235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c09f98-6f35-49d1-bd73-3b63bd9280db None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.137725 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f815b55-5997-487d-8012-333b46f34b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.138588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f815b55-5997-487d-8012-333b46f34b31 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.138918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f815b55-5997-487d-8012-333b46f34b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.139325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f815b55-5997-487d-8012-333b46f34b31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.157784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f815b55-5997-487d-8012-333b46f34b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-5f815b55-5997-487d-8012-333b46f34b31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:18.165720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:18.166695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:18.167492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 134/279] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:18.168157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f815b55-5997-487d-8012-333b46f34b31 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.191646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e428a9-2d99-41c3-8e15-45d63038d379 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.235659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6b836-54d8-4d42-bc90-c071db7c7372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.236590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6b836-54d8-4d42-bc90-c071db7c7372 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.236864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6b836-54d8-4d42-bc90-c071db7c7372 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.237136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6b836-54d8-4d42-bc90-c071db7c7372 None admin] PATH_INFO: `/v3/users/d150ec2538804dc788b00129edbbcd44` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.238384 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6b836-54d8-4d42-bc90-c071db7c7372 None admin] RBAC: Authorizing `identity:get_user(user_id=d150ec2538804dc788b00129edbbcd44)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.239869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6b836-54d8-4d42-bc90-c071db7c7372 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.241951 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 146/280] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/users/d150ec2538804dc788b00129edbbcd44 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:18.242601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6b836-54d8-4d42-bc90-c071db7c7372 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.274907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:18.317256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f815b55-5997-487d-8012-333b46f34b31 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.354426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6b836-54d8-4d42-bc90-c071db7c7372 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.358203 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8e767-8d5b-457b-8e73-cf9360253257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.358891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8e767-8d5b-457b-8e73-cf9360253257 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.359177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8e767-8d5b-457b-8e73-cf9360253257 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.360140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8e767-8d5b-457b-8e73-cf9360253257 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.361239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8e767-8d5b-457b-8e73-cf9360253257 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.362708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8e767-8d5b-457b-8e73-cf9360253257 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.381977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:18.389930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 135/281] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:18 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:18.393067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc3af5-0d6a-4f09-9bdb-b3d1e073f55a None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.393591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc3af5-0d6a-4f09-9bdb-b3d1e073f55a None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.393990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc3af5-0d6a-4f09-9bdb-b3d1e073f55a None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.396545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 136/282] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:18.402161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc3af5-0d6a-4f09-9bdb-b3d1e073f55a None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.423031 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160153cd-9456-482c-a243-6d441e61d968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.424335 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160153cd-9456-482c-a243-6d441e61d968 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.424840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160153cd-9456-482c-a243-6d441e61d968 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.425526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160153cd-9456-482c-a243-6d441e61d968 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.426790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160153cd-9456-482c-a243-6d441e61d968 None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:18.427463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160153cd-9456-482c-a243-6d441e61d968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.430033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160153cd-9456-482c-a243-6d441e61d968 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.453338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 147/283] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:18.453338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd398c25-5fdb-4284-a1f1-21bd84bddc85 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.453338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd398c25-5fdb-4284-a1f1-21bd84bddc85 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.453338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd398c25-5fdb-4284-a1f1-21bd84bddc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.474414 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-fd398c25-5fdb-4284-a1f1-21bd84bddc85 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:18.475745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fd398c25-5fdb-4284-a1f1-21bd84bddc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:18.572502 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 148/284] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:18.572502 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd398c25-5fdb-4284-a1f1-21bd84bddc85 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.575903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc3af5-0d6a-4f09-9bdb-b3d1e073f55a None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.610132 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:18.628930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69058ea1-35e2-4146-9b43-8c7f8ed3ca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.629840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69058ea1-35e2-4146-9b43-8c7f8ed3ca09 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.630122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69058ea1-35e2-4146-9b43-8c7f8ed3ca09 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.630396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69058ea1-35e2-4146-9b43-8c7f8ed3ca09 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.631683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69058ea1-35e2-4146-9b43-8c7f8ed3ca09 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.633402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69058ea1-35e2-4146-9b43-8c7f8ed3ca09 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.635191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 137/285] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:18.636654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69058ea1-35e2-4146-9b43-8c7f8ed3ca09 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.767400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd398c25-5fdb-4284-a1f1-21bd84bddc85 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.774602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:18.834156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982faa9d-acbb-4d58-b9b5-8e1a0042430b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.835642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982faa9d-acbb-4d58-b9b5-8e1a0042430b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.836111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982faa9d-acbb-4d58-b9b5-8e1a0042430b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.836607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982faa9d-acbb-4d58-b9b5-8e1a0042430b None admin] PATH_INFO: `/v3/users/1c6a025a99e949daa5d293d5c1b1fc96` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.846640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69058ea1-35e2-4146-9b43-8c7f8ed3ca09 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:18.865942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982faa9d-acbb-4d58-b9b5-8e1a0042430b None admin] RBAC: Authorizing `identity:get_user(user_id=1c6a025a99e949daa5d293d5c1b1fc96)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:18.869042 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982faa9d-acbb-4d58-b9b5-8e1a0042430b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:18.873495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 149/286] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/users/1c6a025a99e949daa5d293d5c1b1fc96 => generated 293 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:18.874924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982faa9d-acbb-4d58-b9b5-8e1a0042430b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:18.918981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7c989-37d7-404a-a2bb-7a5826b26c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:18.920665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7c989-37d7-404a-a2bb-7a5826b26c4f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:18.921227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7c989-37d7-404a-a2bb-7a5826b26c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:18.922647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7c989-37d7-404a-a2bb-7a5826b26c4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:18.945026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7c989-37d7-404a-a2bb-7a5826b26c4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-48a7c989-37d7-404a-a2bb-7a5826b26c4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:18.961548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:18.964806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:18.967758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:18.967758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:18.967758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:18.967758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:18.967758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:18.967758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:18.967758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:18.967758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:18.967758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:18.967758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:18.967758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:18.967758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:18.980585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 138/287] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:18.980585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7c989-37d7-404a-a2bb-7a5826b26c4f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:19.053298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:19.098277 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982faa9d-acbb-4d58-b9b5-8e1a0042430b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:19.163037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7c989-37d7-404a-a2bb-7a5826b26c4f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:19.164937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c0f19-606a-4633-9ade-a4a9ff964319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:19.166808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c0f19-606a-4633-9ade-a4a9ff964319 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:19.167499 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c0f19-606a-4633-9ade-a4a9ff964319 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:19.168205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c0f19-606a-4633-9ade-a4a9ff964319 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9/users/d150ec2538804dc788b00129edbbcd44/roles/35dbb86a4b774e0a9d862b6fa21cd947` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:19.172804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c0f19-606a-4633-9ade-a4a9ff964319 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f4b4898eca3450daecc84800ff218c9, user_id=d150ec2538804dc788b00129edbbcd44, role_id=35dbb86a4b774e0a9d862b6fa21cd947)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:19.175887 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c0f19-606a-4633-9ade-a4a9ff964319 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:19.194064 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:19.200574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 150/288] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:19:18 2022] PUT /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9/users/d150ec2538804dc788b00129edbbcd44/roles/35dbb86a4b774e0a9d862b6fa21cd947 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:19.201262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c0f19-606a-4633-9ade-a4a9ff964319 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:19.213687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77734f-9be8-4f3d-91d0-6f7b73c2102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:19.214898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77734f-9be8-4f3d-91d0-6f7b73c2102e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:19.216077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77734f-9be8-4f3d-91d0-6f7b73c2102e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:19.216705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77734f-9be8-4f3d-91d0-6f7b73c2102e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:19.220032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77734f-9be8-4f3d-91d0-6f7b73c2102e None admin] RBAC: Adding query filter params (role.id=1b870be20b0c4159bfbbe73593799250, user.id=1c6a025a99e949daa5d293d5c1b1fc96, scope.system=all) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:19.221307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77734f-9be8-4f3d-91d0-6f7b73c2102e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:19.226070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77734f-9be8-4f3d-91d0-6f7b73c2102e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:19.249229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 139/289] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Jul 21 22:19:18 2022] GET /identity/v3/role_assignments?user.id=1c6a025a99e949daa5d293d5c1b1fc96&scope.system=all&role.id=1b870be20b0c4159bfbbe73593799250 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:19.417023 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c0f19-606a-4633-9ade-a4a9ff964319 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:19.501332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58913f35-78e3-4729-8033-8aa3878f8b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:19.502659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58913f35-78e3-4729-8033-8aa3878f8b07 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:19.503118 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58913f35-78e3-4729-8033-8aa3878f8b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:19.503630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58913f35-78e3-4729-8033-8aa3878f8b07 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:19.505293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58913f35-78e3-4729-8033-8aa3878f8b07 None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:19.505848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58913f35-78e3-4729-8033-8aa3878f8b07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:19.508628 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58913f35-78e3-4729-8033-8aa3878f8b07 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:19.519563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:19.528790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 151/290] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:19.537206 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77734f-9be8-4f3d-91d0-6f7b73c2102e None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:19.673700 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77734f-9be8-4f3d-91d0-6f7b73c2102e None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:19.721142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de57be-149d-40bd-928d-b83a3156c31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:19.722038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de57be-149d-40bd-928d-b83a3156c31b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:19.722317 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de57be-149d-40bd-928d-b83a3156c31b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:19.722599 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de57be-149d-40bd-928d-b83a3156c31b None admin] PATH_INFO: `/v3/users/68f50535c0234991b4672e62cfcecdae` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:19.723918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de57be-149d-40bd-928d-b83a3156c31b None admin] RBAC: Authorizing `identity:get_user(user_id=68f50535c0234991b4672e62cfcecdae)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:19.726001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de57be-149d-40bd-928d-b83a3156c31b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:19.729679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 140/291] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:19 2022] GET /identity/v3/users/68f50535c0234991b4672e62cfcecdae => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:19.737420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58913f35-78e3-4729-8033-8aa3878f8b07 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:19.953133 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58913f35-78e3-4729-8033-8aa3878f8b07 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:20.009715 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5920a2f-27bb-4968-8013-be90bad42b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:20.010387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5920a2f-27bb-4968-8013-be90bad42b44 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:20.010670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5920a2f-27bb-4968-8013-be90bad42b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:20.010947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5920a2f-27bb-4968-8013-be90bad42b44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:20.116146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5920a2f-27bb-4968-8013-be90bad42b44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:20.117689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5920a2f-27bb-4968-8013-be90bad42b44 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:20.120927 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:20.277795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 152/292] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:20.284711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de57be-149d-40bd-928d-b83a3156c31b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:20.372893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:20.419130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de57be-149d-40bd-928d-b83a3156c31b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:20.460227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077de4b-0cce-4969-ac81-96315bddbd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:20.461156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077de4b-0cce-4969-ac81-96315bddbd87 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:20.461442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077de4b-0cce-4969-ac81-96315bddbd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:20.461726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077de4b-0cce-4969-ac81-96315bddbd87 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:20.469943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077de4b-0cce-4969-ac81-96315bddbd87 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:20.471429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077de4b-0cce-4969-ac81-96315bddbd87 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:20.473636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 141/293] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:20 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:20.479644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5920a2f-27bb-4968-8013-be90bad42b44 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:20.622989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:20.626682 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5920a2f-27bb-4968-8013-be90bad42b44 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:20.685753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341346fb-e3f8-44f7-ad92-c6d65e5c85ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:20.686435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341346fb-e3f8-44f7-ad92-c6d65e5c85ab None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:20.686719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341346fb-e3f8-44f7-ad92-c6d65e5c85ab None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:20.686998 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341346fb-e3f8-44f7-ad92-c6d65e5c85ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:20.688640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341346fb-e3f8-44f7-ad92-c6d65e5c85ab None admin] RBAC: Adding query filter params (role.id=1b870be20b0c4159bfbbe73593799250, scope.project.id=8f4b4898eca3450daecc84800ff218c9, user.id=68f50535c0234991b4672e62cfcecdae) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:20.689017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341346fb-e3f8-44f7-ad92-c6d65e5c85ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:20.690554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341346fb-e3f8-44f7-ad92-c6d65e5c85ab None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:20.701841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 153/294] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:20 2022] GET /identity/v3/role_assignments?user.id=68f50535c0234991b4672e62cfcecdae&scope.project.id=8f4b4898eca3450daecc84800ff218c9&role.id=1b870be20b0c4159bfbbe73593799250 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:21.893277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e90b9b-e3f2-4b7c-bf22-0960ce02d1f2 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:21.893819 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e90b9b-e3f2-4b7c-bf22-0960ce02d1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:21.894301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e90b9b-e3f2-4b7c-bf22-0960ce02d1f2 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:21.897720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 142/295] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:21.908284 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164cefd-5433-489a-baec-66a04d31b4bf None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:21.909119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164cefd-5433-489a-baec-66a04d31b4bf None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:21.909568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164cefd-5433-489a-baec-66a04d31b4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:21.936268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d164cefd-5433-489a-baec-66a04d31b4bf None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:21.938665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d164cefd-5433-489a-baec-66a04d31b4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:21.986022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd048a9-244e-4c43-bd64-2c9857adef59 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:21.986526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd048a9-244e-4c43-bd64-2c9857adef59 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:21.987015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd048a9-244e-4c43-bd64-2c9857adef59 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:21.993432 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 143/296] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:22.000655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214d23e-b922-4748-a306-cc5a27f3cb05 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.001171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214d23e-b922-4748-a306-cc5a27f3cb05 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.001675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214d23e-b922-4748-a306-cc5a27f3cb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.028893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-b214d23e-b922-4748-a306-cc5a27f3cb05 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:22.031550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b214d23e-b922-4748-a306-cc5a27f3cb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:22.034405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 154/297] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:22.078977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164cefd-5433-489a-baec-66a04d31b4bf None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:22.123152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 144/298] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:22.164901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214d23e-b922-4748-a306-cc5a27f3cb05 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:22.208232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:22.217975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164cefd-5433-489a-baec-66a04d31b4bf None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:22.268205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6fcab-af82-4e30-aca7-14ed7962b84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:22.269105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6fcab-af82-4e30-aca7-14ed7962b84e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.269390 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6fcab-af82-4e30-aca7-14ed7962b84e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.269668 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6fcab-af82-4e30-aca7-14ed7962b84e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.271125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6fcab-af82-4e30-aca7-14ed7962b84e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:22.273480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6fcab-af82-4e30-aca7-14ed7962b84e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:22.275379 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 155/299] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:22.276051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:22.276494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256664fe-02b5-48f9-bc34-b82329e949f4 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.277167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256664fe-02b5-48f9-bc34-b82329e949f4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.277167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256664fe-02b5-48f9-bc34-b82329e949f4 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.278542 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 156/300] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:22.284279 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256664fe-02b5-48f9-bc34-b82329e949f4 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:22.339149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214d23e-b922-4748-a306-cc5a27f3cb05 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:22.391484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb217b1-5dff-40a6-9336-b592324bb5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:22.392310 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb217b1-5dff-40a6-9336-b592324bb5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.392650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb217b1-5dff-40a6-9336-b592324bb5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.392933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb217b1-5dff-40a6-9336-b592324bb5e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.404789 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb217b1-5dff-40a6-9336-b592324bb5e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-3cb217b1-5dff-40a6-9336-b592324bb5e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:22.412896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:22.414607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 145/301] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:22.416095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151600-b5fd-42a1-b5df-05da5eff2117 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.417441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151600-b5fd-42a1-b5df-05da5eff2117 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.417441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151600-b5fd-42a1-b5df-05da5eff2117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.437001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-59151600-b5fd-42a1-b5df-05da5eff2117 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:22.438966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-59151600-b5fd-42a1-b5df-05da5eff2117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:22.443368 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256664fe-02b5-48f9-bc34-b82329e949f4 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:22.512323 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d947c6-ec1e-4908-81a3-0543714d8fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:22.513528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d947c6-ec1e-4908-81a3-0543714d8fe9 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.514047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d947c6-ec1e-4908-81a3-0543714d8fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.514566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d947c6-ec1e-4908-81a3-0543714d8fe9 None admin] PATH_INFO: `/v3/groups` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.515917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d947c6-ec1e-4908-81a3-0543714d8fe9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:22.517539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d947c6-ec1e-4908-81a3-0543714d8fe9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:22.520333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 146/302] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:22.521260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59151600-b5fd-42a1-b5df-05da5eff2117 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:22.544161 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 157/303] 10.222.0.42 () {60 vars in 1134 bytes} [Thu Jul 21 22:19:22 2022] POST /identity/v3/groups => generated 229 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:19:22.547710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f130ac2-5cc8-4c7d-9a74-c130f32747ed None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.548526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f130ac2-5cc8-4c7d-9a74-c130f32747ed None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.549120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f130ac2-5cc8-4c7d-9a74-c130f32747ed None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.552134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 158/304] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:22.561652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e201ab4-de8d-4460-9f4d-0120568388d7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.563184 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e201ab4-de8d-4460-9f4d-0120568388d7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.564028 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e201ab4-de8d-4460-9f4d-0120568388d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.594662 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4e201ab4-de8d-4460-9f4d-0120568388d7 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:22.595878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:22.597295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4e201ab4-de8d-4460-9f4d-0120568388d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:22.637981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:22.701260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59151600-b5fd-42a1-b5df-05da5eff2117 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:22.757860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 159/305] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:22.759354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e201ab4-de8d-4460-9f4d-0120568388d7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:22.778715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f917a5e-0f20-4c54-8f89-863523f9218a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:22.779750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f917a5e-0f20-4c54-8f89-863523f9218a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:22.780250 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f917a5e-0f20-4c54-8f89-863523f9218a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:22.780902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f917a5e-0f20-4c54-8f89-863523f9218a None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:22.784278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f917a5e-0f20-4c54-8f89-863523f9218a None admin] RBAC: Adding query filter params (name=member) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:22.786349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f917a5e-0f20-4c54-8f89-863523f9218a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:22.786349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f917a5e-0f20-4c54-8f89-863523f9218a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:22.798830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 147/306] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:22.800282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f917a5e-0f20-4c54-8f89-863523f9218a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:22.957140 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e201ab4-de8d-4460-9f4d-0120568388d7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:22.990393 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f917a5e-0f20-4c54-8f89-863523f9218a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.018163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:23.035587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469da471-9f3a-4985-b9f4-12a7d6b36ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.036659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469da471-9f3a-4985-b9f4-12a7d6b36ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.036945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469da471-9f3a-4985-b9f4-12a7d6b36ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.037216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469da471-9f3a-4985-b9f4-12a7d6b36ca8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.057878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469da471-9f3a-4985-b9f4-12a7d6b36ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:23.070202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9280c-a6f6-4e78-8f72-3ee0866d08cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-469da471-9f3a-4985-b9f4-12a7d6b36ca8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:23.071248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:23.081199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:23.082679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:23.082679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:23.082679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:23.082679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:23.082679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:23.082679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:23.082679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:23.082679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:23.082679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:23.082679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:23.082679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:23.082679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:23.083927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9280c-a6f6-4e78-8f72-3ee0866d08cd None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.084088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 160/307] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:23.084239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9280c-a6f6-4e78-8f72-3ee0866d08cd None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.084239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9280c-a6f6-4e78-8f72-3ee0866d08cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.084566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469da471-9f3a-4985-b9f4-12a7d6b36ca8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:23.104536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9280c-a6f6-4e78-8f72-3ee0866d08cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:23.107941 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-72a9280c-a6f6-4e78-8f72-3ee0866d08cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:23.121173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:23.122321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:23.123748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 148/308] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:23.125203 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9280c-a6f6-4e78-8f72-3ee0866d08cd None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:23.250689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469da471-9f3a-4985-b9f4-12a7d6b36ca8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.312260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9280c-a6f6-4e78-8f72-3ee0866d08cd None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.328530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792cfcc-2883-40ac-832d-e6a9735bf7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.330416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792cfcc-2883-40ac-832d-e6a9735bf7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.331134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792cfcc-2883-40ac-832d-e6a9735bf7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.331883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792cfcc-2883-40ac-832d-e6a9735bf7b9 None admin] PATH_INFO: `/v3/users/1c6a025a99e949daa5d293d5c1b1fc96` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.334190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792cfcc-2883-40ac-832d-e6a9735bf7b9 None admin] RBAC: Authorizing `identity:get_user(user_id=1c6a025a99e949daa5d293d5c1b1fc96)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:23.336978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792cfcc-2883-40ac-832d-e6a9735bf7b9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:23.345785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 161/309] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:23 2022] GET /identity/v3/users/1c6a025a99e949daa5d293d5c1b1fc96 => generated 293 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:23.345785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792cfcc-2883-40ac-832d-e6a9735bf7b9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:23.367964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b656f7-fcd3-4f53-9cc1-68d406444131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.368836 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b656f7-fcd3-4f53-9cc1-68d406444131 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.369187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b656f7-fcd3-4f53-9cc1-68d406444131 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.369614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b656f7-fcd3-4f53-9cc1-68d406444131 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.370665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b656f7-fcd3-4f53-9cc1-68d406444131 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:23.371108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b656f7-fcd3-4f53-9cc1-68d406444131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:23.373577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b656f7-fcd3-4f53-9cc1-68d406444131 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:23.395327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 149/310] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:23.395327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b656f7-fcd3-4f53-9cc1-68d406444131 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:23.454988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:23.517249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:23.550944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792cfcc-2883-40ac-832d-e6a9735bf7b9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.553989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b656f7-fcd3-4f53-9cc1-68d406444131 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.616425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638994c-c106-401f-b71b-375f4420a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.617915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d29df8-7828-450e-bfb8-8a349b3ca077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.618068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638994c-c106-401f-b71b-375f4420a2d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.618446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638994c-c106-401f-b71b-375f4420a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.618959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638994c-c106-401f-b71b-375f4420a2d4 None admin] PATH_INFO: `/v3/system/users/1c6a025a99e949daa5d293d5c1b1fc96/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.619305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d29df8-7828-450e-bfb8-8a349b3ca077 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.619772 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d29df8-7828-450e-bfb8-8a349b3ca077 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.620339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d29df8-7828-450e-bfb8-8a349b3ca077 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.621781 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638994c-c106-401f-b71b-375f4420a2d4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1c6a025a99e949daa5d293d5c1b1fc96, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:23.622350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d29df8-7828-450e-bfb8-8a349b3ca077 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:23.622858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d29df8-7828-450e-bfb8-8a349b3ca077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:23.623283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638994c-c106-401f-b71b-375f4420a2d4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:23.625204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d29df8-7828-450e-bfb8-8a349b3ca077 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:23.639828 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 150/311] 10.222.0.42 () {58 vars in 1334 bytes} [Thu Jul 21 22:19:23 2022] PUT /identity/v3/system/users/1c6a025a99e949daa5d293d5c1b1fc96/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:23.642178 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638994c-c106-401f-b71b-375f4420a2d4 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:23.643008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 162/312] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:23.682766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d29df8-7828-450e-bfb8-8a349b3ca077 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:23.826862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:23.834548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638994c-c106-401f-b71b-375f4420a2d4 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.885805 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7967c9-afcd-42a4-b774-b5b929497ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.886719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7967c9-afcd-42a4-b774-b5b929497ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.887008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7967c9-afcd-42a4-b774-b5b929497ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.887290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7967c9-afcd-42a4-b774-b5b929497ec5 None admin] PATH_INFO: `/v3/users/d150ec2538804dc788b00129edbbcd44` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.888687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7967c9-afcd-42a4-b774-b5b929497ec5 None admin] RBAC: Authorizing `identity:get_user(user_id=d150ec2538804dc788b00129edbbcd44)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:23.890706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7967c9-afcd-42a4-b774-b5b929497ec5 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:23.894089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 151/313] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:23 2022] GET /identity/v3/users/d150ec2538804dc788b00129edbbcd44 => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:23.898599 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d29df8-7828-450e-bfb8-8a349b3ca077 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:23.900521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7967c9-afcd-42a4-b774-b5b929497ec5 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:23.949252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1017433-46ae-429e-b18e-9e47e464a268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:23.950842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1017433-46ae-429e-b18e-9e47e464a268 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:23.951364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1017433-46ae-429e-b18e-9e47e464a268 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:23.951784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1017433-46ae-429e-b18e-9e47e464a268 None admin] PATH_INFO: `/v3/users/ee4895e470d54a038729ec1c12b7cd22` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:23.975110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:23.979326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1017433-46ae-429e-b18e-9e47e464a268 None admin] RBAC: Authorizing `identity:get_user(user_id=ee4895e470d54a038729ec1c12b7cd22)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:23.982440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1017433-46ae-429e-b18e-9e47e464a268 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:23.986893 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 163/314] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:23 2022] GET /identity/v3/users/ee4895e470d54a038729ec1c12b7cd22 => generated 293 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:23.993761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1017433-46ae-429e-b18e-9e47e464a268 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:24.051082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7967c9-afcd-42a4-b774-b5b929497ec5 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:24.106860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a9776-f53b-4379-b027-12540fde9ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:24.108510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a9776-f53b-4379-b027-12540fde9ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.108805 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a9776-f53b-4379-b027-12540fde9ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.109694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a9776-f53b-4379-b027-12540fde9ee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.172348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:24.175102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1017433-46ae-429e-b18e-9e47e464a268 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:24.235777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a9776-f53b-4379-b027-12540fde9ee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:24.238966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a9776-f53b-4379-b027-12540fde9ee4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:24.246537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702dc7d-3261-49aa-b219-0f3393029f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:24.247860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702dc7d-3261-49aa-b219-0f3393029f06 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.248491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702dc7d-3261-49aa-b219-0f3393029f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.249133 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702dc7d-3261-49aa-b219-0f3393029f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.251038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702dc7d-3261-49aa-b219-0f3393029f06 None admin] RBAC: Adding query filter params (role.id=35dbb86a4b774e0a9d862b6fa21cd947, user.id=ee4895e470d54a038729ec1c12b7cd22, scope.system=all) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:24.251502 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702dc7d-3261-49aa-b219-0f3393029f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:24.253474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702dc7d-3261-49aa-b219-0f3393029f06 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:24.269175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 164/315] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Jul 21 22:19:23 2022] GET /identity/v3/role_assignments?user.id=ee4895e470d54a038729ec1c12b7cd22&scope.system=all&role.id=35dbb86a4b774e0a9d862b6fa21cd947 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:24.394617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 152/316] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:19:24.400937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702dc7d-3261-49aa-b219-0f3393029f06 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:24.516441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:24.586444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702dc7d-3261-49aa-b219-0f3393029f06 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:24.625227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ba9ab-aaa8-477f-b4c3-28186184dc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:24.626105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ba9ab-aaa8-477f-b4c3-28186184dc5e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.626454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ba9ab-aaa8-477f-b4c3-28186184dc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.626731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ba9ab-aaa8-477f-b4c3-28186184dc5e None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.628027 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ba9ab-aaa8-477f-b4c3-28186184dc5e None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:24.629566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ba9ab-aaa8-477f-b4c3-28186184dc5e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:24.636301 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 165/317] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:24 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:24.642508 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a9776-f53b-4379-b027-12540fde9ee4 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:24.719165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:24.768514 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a9776-f53b-4379-b027-12540fde9ee4 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:24.809604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7c56d-4f5c-4d88-b0ce-e3b4c50320b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:24.810643 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7c56d-4f5c-4d88-b0ce-e3b4c50320b0 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.811106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7c56d-4f5c-4d88-b0ce-e3b4c50320b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.811559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7c56d-4f5c-4d88-b0ce-e3b4c50320b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.813492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7c56d-4f5c-4d88-b0ce-e3b4c50320b0 None admin] RBAC: Adding query filter params (role.id=35dbb86a4b774e0a9d862b6fa21cd947, scope.project.id=8f4b4898eca3450daecc84800ff218c9, user.id=d150ec2538804dc788b00129edbbcd44) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:24.814035 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7c56d-4f5c-4d88-b0ce-e3b4c50320b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:24.815699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7c56d-4f5c-4d88-b0ce-e3b4c50320b0 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:24.828284 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 153/318] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:24 2022] GET /identity/v3/role_assignments?user.id=d150ec2538804dc788b00129edbbcd44&scope.project.id=8f4b4898eca3450daecc84800ff218c9&role.id=35dbb86a4b774e0a9d862b6fa21cd947 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:24.964565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fe81b-943e-4688-bbaf-78cdefb9c1fd None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.965244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fe81b-943e-4688-bbaf-78cdefb9c1fd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.965747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fe81b-943e-4688-bbaf-78cdefb9c1fd None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:24.967866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 166/319] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:24.981622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d5c35-c472-4a92-8e1a-b20edb6f2f82 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:24.981908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d5c35-c472-4a92-8e1a-b20edb6f2f82 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:24.982111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d5c35-c472-4a92-8e1a-b20edb6f2f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.002728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-424d5c35-c472-4a92-8e1a-b20edb6f2f82 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:25.004061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-424d5c35-c472-4a92-8e1a-b20edb6f2f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:25.121958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 154/320] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:25.145472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fe81b-943e-4688-bbaf-78cdefb9c1fd None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:25.314060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fe81b-943e-4688-bbaf-78cdefb9c1fd None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:25.360132 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:25.368730 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176eac40-2f92-49d9-976c-f8234e18fbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:25.369974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eac40-2f92-49d9-976c-f8234e18fbda None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.370498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eac40-2f92-49d9-976c-f8234e18fbda None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.371012 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eac40-2f92-49d9-976c-f8234e18fbda None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.373231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176eac40-2f92-49d9-976c-f8234e18fbda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:25.375919 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176eac40-2f92-49d9-976c-f8234e18fbda None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:25.384587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 167/321] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:25.386089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d5c35-c472-4a92-8e1a-b20edb6f2f82 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:25.493318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:25.552752 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d5c35-c472-4a92-8e1a-b20edb6f2f82 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:25.607982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcc9b0-83bb-4b72-9819-ca1f0b94bcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:25.608590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcc9b0-83bb-4b72-9819-ca1f0b94bcb1 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.609798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcc9b0-83bb-4b72-9819-ca1f0b94bcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.609798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcc9b0-83bb-4b72-9819-ca1f0b94bcb1 None admin] PATH_INFO: `/v3/groups` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.610483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fcc9b0-83bb-4b72-9819-ca1f0b94bcb1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:25.611934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fcc9b0-83bb-4b72-9819-ca1f0b94bcb1 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:25.633017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 155/322] 10.222.0.42 () {60 vars in 1134 bytes} [Thu Jul 21 22:19:25 2022] POST /identity/v3/groups => generated 226 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:25.865051 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2724f-4710-4711-a490-0220a5199943 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.865355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2724f-4710-4711-a490-0220a5199943 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.865535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2724f-4710-4711-a490-0220a5199943 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.866950 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 168/323] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:19:25.874277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6f354-a035-46a5-a5fb-537a9f0eb815 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:25.875238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6f354-a035-46a5-a5fb-537a9f0eb815 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:25.875592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6f354-a035-46a5-a5fb-537a9f0eb815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:25.895698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-79a6f354-a035-46a5-a5fb-537a9f0eb815 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:25.897469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-79a6f354-a035-46a5-a5fb-537a9f0eb815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:25.988960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 156/324] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:26.012781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2724f-4710-4711-a490-0220a5199943 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.138061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2724f-4710-4711-a490-0220a5199943 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:26.217874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd448f2-232d-4a2e-8b15-fc464864318a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:26.219456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd448f2-232d-4a2e-8b15-fc464864318a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.219607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd448f2-232d-4a2e-8b15-fc464864318a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.224512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd448f2-232d-4a2e-8b15-fc464864318a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.235789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd448f2-232d-4a2e-8b15-fc464864318a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-cdd448f2-232d-4a2e-8b15-fc464864318a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:26.243827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:26.244851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:26.245752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:26.245752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:26.245752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:26.245752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:26.245752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:26.245752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:26.245752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:26.245752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:26.245752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:26.245752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:26.245752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:26.245752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:26.247151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 169/325] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:26.256288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6f354-a035-46a5-a5fb-537a9f0eb815 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.339834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:26.472405 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6f354-a035-46a5-a5fb-537a9f0eb815 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:26.542167 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41d566-69f9-4b35-9b08-2cef7363a540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:26.543340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41d566-69f9-4b35-9b08-2cef7363a540 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.543825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41d566-69f9-4b35-9b08-2cef7363a540 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.544320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41d566-69f9-4b35-9b08-2cef7363a540 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.545805 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41d566-69f9-4b35-9b08-2cef7363a540 None admin] RBAC: Adding query filter params (name=member) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:26.546456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41d566-69f9-4b35-9b08-2cef7363a540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:26.549172 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41d566-69f9-4b35-9b08-2cef7363a540 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:26.553819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8dde2-24ee-4638-add1-90e4ef264ff4 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.554158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8dde2-24ee-4638-add1-90e4ef264ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.554417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8dde2-24ee-4638-add1-90e4ef264ff4 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.556042 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 170/326] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:26.561549 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29da95a-ad5a-4e70-8177-ddf44365385a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.562421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29da95a-ad5a-4e70-8177-ddf44365385a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.562695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29da95a-ad5a-4e70-8177-ddf44365385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.568190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 157/327] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:26.583739 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41d566-69f9-4b35-9b08-2cef7363a540 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.604763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f29da95a-ad5a-4e70-8177-ddf44365385a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:26.606543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f29da95a-ad5a-4e70-8177-ddf44365385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:26.693477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:26.722416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 171/328] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:26.770615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29da95a-ad5a-4e70-8177-ddf44365385a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.793624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41d566-69f9-4b35-9b08-2cef7363a540 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:26.819102 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:26.873210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176b03-ef5f-4043-b3e5-ddd625040124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:26.874554 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176b03-ef5f-4043-b3e5-ddd625040124 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:26.876828 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176b03-ef5f-4043-b3e5-ddd625040124 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:26.877368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176b03-ef5f-4043-b3e5-ddd625040124 None admin] PATH_INFO: `/v3/users/1c6a025a99e949daa5d293d5c1b1fc96` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:26.880025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5176b03-ef5f-4043-b3e5-ddd625040124 None admin] RBAC: Authorizing `identity:get_user(user_id=1c6a025a99e949daa5d293d5c1b1fc96)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:26.881807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5176b03-ef5f-4043-b3e5-ddd625040124 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:26.884203 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 158/329] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:26 2022] GET /identity/v3/users/1c6a025a99e949daa5d293d5c1b1fc96 => generated 293 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:26.903949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176b03-ef5f-4043-b3e5-ddd625040124 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:26.968255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29da95a-ad5a-4e70-8177-ddf44365385a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:27.030131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29812dd-3ba0-4946-9a1d-38556f7eaf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:27.031603 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29812dd-3ba0-4946-9a1d-38556f7eaf13 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:27.032109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29812dd-3ba0-4946-9a1d-38556f7eaf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:27.032656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29812dd-3ba0-4946-9a1d-38556f7eaf13 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:27.074359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29812dd-3ba0-4946-9a1d-38556f7eaf13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-f29812dd-3ba0-4946-9a1d-38556f7eaf13 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:27.084882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:27.085846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:27.086629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:27.086629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:27.086629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:27.086629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:27.086629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:27.086629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:27.086629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:27.086629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:27.086629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:27.086629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:27.086629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:27.086629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:27.092121 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 172/330] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:27.099335 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29812dd-3ba0-4946-9a1d-38556f7eaf13 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:27.244713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:27.259078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176b03-ef5f-4043-b3e5-ddd625040124 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:27.321405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29812dd-3ba0-4946-9a1d-38556f7eaf13 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:27.372409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018fd4b-8824-4a5a-84e4-3a7fb103e23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:27.373471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018fd4b-8824-4a5a-84e4-3a7fb103e23c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:27.373916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018fd4b-8824-4a5a-84e4-3a7fb103e23c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:27.374366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018fd4b-8824-4a5a-84e4-3a7fb103e23c None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:27.376254 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018fd4b-8824-4a5a-84e4-3a7fb103e23c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:27.376754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018fd4b-8824-4a5a-84e4-3a7fb103e23c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:27.378551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018fd4b-8824-4a5a-84e4-3a7fb103e23c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:27.393542 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 173/331] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:27.399239 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018fd4b-8824-4a5a-84e4-3a7fb103e23c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:27.427006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc041a9-7188-4e87-952a-d3c2562a0e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:27.428064 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc041a9-7188-4e87-952a-d3c2562a0e18 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:27.428716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc041a9-7188-4e87-952a-d3c2562a0e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:27.429346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc041a9-7188-4e87-952a-d3c2562a0e18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:27.431190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc041a9-7188-4e87-952a-d3c2562a0e18 None admin] RBAC: Adding query filter params (role.id=1b870be20b0c4159bfbbe73593799250, user.id=1c6a025a99e949daa5d293d5c1b1fc96, scope.system=all) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:27.431803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc041a9-7188-4e87-952a-d3c2562a0e18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:27.439532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc041a9-7188-4e87-952a-d3c2562a0e18 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:27.499077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 159/332] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Jul 21 22:19:26 2022] GET /identity/v3/role_assignments?user.id=1c6a025a99e949daa5d293d5c1b1fc96&scope.system=all&role.id=1b870be20b0c4159bfbbe73593799250 => generated 510 bytes in 595 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:27.557905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018fd4b-8824-4a5a-84e4-3a7fb103e23c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:27.575682 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:27.604857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a662a-023f-4c24-bdc5-6baedbc98f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:27.605744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a662a-023f-4c24-bdc5-6baedbc98f77 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:27.606036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a662a-023f-4c24-bdc5-6baedbc98f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:27.606317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a662a-023f-4c24-bdc5-6baedbc98f77 None admin] PATH_INFO: `/v3/users/ee4895e470d54a038729ec1c12b7cd22` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:27.607561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a662a-023f-4c24-bdc5-6baedbc98f77 None admin] RBAC: Authorizing `identity:get_user(user_id=ee4895e470d54a038729ec1c12b7cd22)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:27.609833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a662a-023f-4c24-bdc5-6baedbc98f77 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:27.611907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 174/333] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:27 2022] GET /identity/v3/users/ee4895e470d54a038729ec1c12b7cd22 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:27.618148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc041a9-7188-4e87-952a-d3c2562a0e18 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:27.698742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:27.794754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc041a9-7188-4e87-952a-d3c2562a0e18 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:27.841715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d669ad-d394-41b4-a899-cba4d7620179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:27.842621 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d669ad-d394-41b4-a899-cba4d7620179 None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:27.842917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d669ad-d394-41b4-a899-cba4d7620179 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:27.843826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d669ad-d394-41b4-a899-cba4d7620179 None admin] PATH_INFO: `/v3/system/users/ee4895e470d54a038729ec1c12b7cd22/roles/35dbb86a4b774e0a9d862b6fa21cd947` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:27.847626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d669ad-d394-41b4-a899-cba4d7620179 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ee4895e470d54a038729ec1c12b7cd22, role_id=35dbb86a4b774e0a9d862b6fa21cd947)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:27.849089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d669ad-d394-41b4-a899-cba4d7620179 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:27.865609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 160/334] 10.222.0.42 () {58 vars in 1334 bytes} [Thu Jul 21 22:19:27 2022] PUT /identity/v3/system/users/ee4895e470d54a038729ec1c12b7cd22/roles/35dbb86a4b774e0a9d862b6fa21cd947 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:19:28.886696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4d6a5-7b5b-4341-a3f1-55af6e51f2ab None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:28.887401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4d6a5-7b5b-4341-a3f1-55af6e51f2ab None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:28.887686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4d6a5-7b5b-4341-a3f1-55af6e51f2ab None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:28.891256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 175/335] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:28.897903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd685a84-5c37-41c5-98ea-9e19001daed6 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:28.898645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd685a84-5c37-41c5-98ea-9e19001daed6 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:28.898840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd685a84-5c37-41c5-98ea-9e19001daed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:28.937246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-bd685a84-5c37-41c5-98ea-9e19001daed6 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:28.939652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bd685a84-5c37-41c5-98ea-9e19001daed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:29.074639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 161/336] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:29.113604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4d6a5-7b5b-4341-a3f1-55af6e51f2ab None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:29.289529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4d6a5-7b5b-4341-a3f1-55af6e51f2ab None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:29.339483 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28177d92-c92c-4224-af9f-aaab18d6f4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:29.340336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28177d92-c92c-4224-af9f-aaab18d6f4bc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.340688 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28177d92-c92c-4224-af9f-aaab18d6f4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.340975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28177d92-c92c-4224-af9f-aaab18d6f4bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.357436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28177d92-c92c-4224-af9f-aaab18d6f4bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-28177d92-c92c-4224-af9f-aaab18d6f4bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:29.367278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:29.369397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:29.371131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 176/337] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:29.376597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd685a84-5c37-41c5-98ea-9e19001daed6 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:29.456193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:29.564303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd685a84-5c37-41c5-98ea-9e19001daed6 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:29.614255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e16e2-49b2-4eb4-93ef-34458a600802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:29.614937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e16e2-49b2-4eb4-93ef-34458a600802 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.615219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e16e2-49b2-4eb4-93ef-34458a600802 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.615497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e16e2-49b2-4eb4-93ef-34458a600802 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.616276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e16e2-49b2-4eb4-93ef-34458a600802 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:29.616629 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e16e2-49b2-4eb4-93ef-34458a600802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:29.618106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e16e2-49b2-4eb4-93ef-34458a600802 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:29.633327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 162/338] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:29.639944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28177d92-c92c-4224-af9f-aaab18d6f4bc None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:29.734737 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:29.796894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31cdc1-9712-4b01-9dd7-85f7d8b71078 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.797765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31cdc1-9712-4b01-9dd7-85f7d8b71078 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.798291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31cdc1-9712-4b01-9dd7-85f7d8b71078 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.799960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 163/339] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:19:29.808516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925db3f0-c1de-4d36-9187-7e994effa7ae None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.809149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925db3f0-c1de-4d36-9187-7e994effa7ae None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.809623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925db3f0-c1de-4d36-9187-7e994effa7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:29.854516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-925db3f0-c1de-4d36-9187-7e994effa7ae None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:29.857243 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-925db3f0-c1de-4d36-9187-7e994effa7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:29.860676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28177d92-c92c-4224-af9f-aaab18d6f4bc None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:29.933229 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9463ae8-15a3-4a54-aa8c-c09f419b38ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:29.935878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9463ae8-15a3-4a54-aa8c-c09f419b38ee None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:29.936617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9463ae8-15a3-4a54-aa8c-c09f419b38ee None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:29.938683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9463ae8-15a3-4a54-aa8c-c09f419b38ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:30.018482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 164/340] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:30.099001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925db3f0-c1de-4d36-9187-7e994effa7ae None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:30.121945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9463ae8-15a3-4a54-aa8c-c09f419b38ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:30.124193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9463ae8-15a3-4a54-aa8c-c09f419b38ee None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:30.169100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:30.253053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:30.318941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925db3f0-c1de-4d36-9187-7e994effa7ae None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:30.381341 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3ee7d-079c-4132-81ef-7c1ddf77268c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:30.382447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3ee7d-079c-4132-81ef-7c1ddf77268c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:30.382895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3ee7d-079c-4132-81ef-7c1ddf77268c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:30.383339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3ee7d-079c-4132-81ef-7c1ddf77268c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:30.402226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 177/341] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 764 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:30.404720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3ee7d-079c-4132-81ef-7c1ddf77268c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:30.409255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9463ae8-15a3-4a54-aa8c-c09f419b38ee None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-ffd3ee7d-079c-4132-81ef-7c1ddf77268c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:30.416511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:30.418410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:30.420164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 165/342] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:30.449635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3ee7d-079c-4132-81ef-7c1ddf77268c None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:30.674302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:30.695141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9463ae8-15a3-4a54-aa8c-c09f419b38ee None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:30.744910 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2566-8d73-4aa8-987a-b5de8e56fad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:30.746163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2566-8d73-4aa8-987a-b5de8e56fad2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:30.746431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2566-8d73-4aa8-987a-b5de8e56fad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:30.746707 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2566-8d73-4aa8-987a-b5de8e56fad2 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:30.747639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3ee7d-079c-4132-81ef-7c1ddf77268c None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:30.748043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e2566-8d73-4aa8-987a-b5de8e56fad2 None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:30.751993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e2566-8d73-4aa8-987a-b5de8e56fad2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:30.755125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 178/343] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:30 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:30.760992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2566-8d73-4aa8-987a-b5de8e56fad2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:30.784711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:30.846355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e898c-8f0b-4508-9116-25c705decbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:30.847080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e898c-8f0b-4508-9116-25c705decbdf None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:30.847371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e898c-8f0b-4508-9116-25c705decbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:30.847655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e898c-8f0b-4508-9116-25c705decbdf None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:30.848451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e898c-8f0b-4508-9116-25c705decbdf None admin] RBAC: Adding query filter params (name=member) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:30.848828 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e898c-8f0b-4508-9116-25c705decbdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:30.850238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e898c-8f0b-4508-9116-25c705decbdf None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:30.871094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 166/344] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:30.880466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e898c-8f0b-4508-9116-25c705decbdf None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:30.926597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2566-8d73-4aa8-987a-b5de8e56fad2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:30.969525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c6bd9-44f5-45c0-9a05-7836204cf404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:30.970718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c6bd9-44f5-45c0-9a05-7836204cf404 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:30.970965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c6bd9-44f5-45c0-9a05-7836204cf404 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:30.971236 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c6bd9-44f5-45c0-9a05-7836204cf404 None admin] PATH_INFO: `/v3/groups/f2c406148932437fa57d1eed86c91879` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:30.978910 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c6bd9-44f5-45c0-9a05-7836204cf404 None admin] RBAC: Authorizing `identity:get_group(group_id=f2c406148932437fa57d1eed86c91879)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:30.980405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c6bd9-44f5-45c0-9a05-7836204cf404 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:30.982975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 179/345] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:30 2022] GET /identity/v3/groups/f2c406148932437fa57d1eed86c91879 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:30.989028 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c6bd9-44f5-45c0-9a05-7836204cf404 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:31.069473 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:31.100603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e898c-8f0b-4508-9116-25c705decbdf None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:31.107175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c6bd9-44f5-45c0-9a05-7836204cf404 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:31.138932 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5b4c8-bf33-49b2-a133-9a3129f8545d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:31.139923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5b4c8-bf33-49b2-a133-9a3129f8545d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.140871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5b4c8-bf33-49b2-a133-9a3129f8545d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.141322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5b4c8-bf33-49b2-a133-9a3129f8545d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.145756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fb7fe-2eac-4384-a955-ba4f42aff25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:31.146705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fb7fe-2eac-4384-a955-ba4f42aff25b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.147389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fb7fe-2eac-4384-a955-ba4f42aff25b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.147840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fb7fe-2eac-4384-a955-ba4f42aff25b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.148910 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fb7fe-2eac-4384-a955-ba4f42aff25b None admin] RBAC: Adding query filter params (group.id=f2c406148932437fa57d1eed86c91879, role.id=80eea410deca46e2924f27afe5136d55, scope.project.id=bd637189b43c45c28839d066df463bc3) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:31.149388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fb7fe-2eac-4384-a955-ba4f42aff25b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:31.151039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fb7fe-2eac-4384-a955-ba4f42aff25b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:31.160630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 180/346] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:30 2022] GET /identity/v3/role_assignments?group.id=f2c406148932437fa57d1eed86c91879&scope.project.id=bd637189b43c45c28839d066df463bc3&role.id=80eea410deca46e2924f27afe5136d55 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:31.160850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:31.163276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655c357-b06e-46cc-b83f-378a295be7f8 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.163757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655c357-b06e-46cc-b83f-378a295be7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.164177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655c357-b06e-46cc-b83f-378a295be7f8 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.166872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 181/347] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:31.168529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a3348-7412-4d29-b0c1-cbfef668cd03 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.168990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a3348-7412-4d29-b0c1-cbfef668cd03 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.169434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a3348-7412-4d29-b0c1-cbfef668cd03 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.174401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 182/348] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:31.180877 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f723ab-9335-4f29-87df-54b64d412545 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.181415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f723ab-9335-4f29-87df-54b64d412545 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.181845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f723ab-9335-4f29-87df-54b64d412545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.217743 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-26f723ab-9335-4f29-87df-54b64d412545 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:31.219760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-26f723ab-9335-4f29-87df-54b64d412545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:31.273341 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5b4c8-bf33-49b2-a133-9a3129f8545d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:31.278874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5b4c8-bf33-49b2-a133-9a3129f8545d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:31.355553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 183/349] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:31.361080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad72d7e-ac7b-4292-8a15-357f4cea911b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.361374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad72d7e-ac7b-4292-8a15-357f4cea911b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.361646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad72d7e-ac7b-4292-8a15-357f4cea911b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.387698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8ad72d7e-ac7b-4292-8a15-357f4cea911b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:31.393409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8ad72d7e-ac7b-4292-8a15-357f4cea911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:31.473310 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 167/350] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:19:31.474198 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5b4c8-bf33-49b2-a133-9a3129f8545d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:31.492991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:31.503735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 184/351] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:31.504455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad72d7e-ac7b-4292-8a15-357f4cea911b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:31.564892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:31.621315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad72d7e-ac7b-4292-8a15-357f4cea911b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:31.649833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5b4c8-bf33-49b2-a133-9a3129f8545d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:31.662450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772deff2-077f-433e-a4ca-ff9d0be394b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:31.663334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772deff2-077f-433e-a4ca-ff9d0be394b5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.663607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772deff2-077f-433e-a4ca-ff9d0be394b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.663878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772deff2-077f-433e-a4ca-ff9d0be394b5 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.665163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772deff2-077f-433e-a4ca-ff9d0be394b5 None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:31.666632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772deff2-077f-433e-a4ca-ff9d0be394b5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:31.668667 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 185/352] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:31 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:31.669323 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772deff2-077f-433e-a4ca-ff9d0be394b5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:31.711144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed43ea-1c70-4a7b-a71f-ca303da033c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:31.711882 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed43ea-1c70-4a7b-a71f-ca303da033c4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.712168 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed43ea-1c70-4a7b-a71f-ca303da033c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.712425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed43ea-1c70-4a7b-a71f-ca303da033c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.728815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed43ea-1c70-4a7b-a71f-ca303da033c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-c9ed43ea-1c70-4a7b-a71f-ca303da033c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:31.738471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:31.741997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 168/353] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:31.751100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed43ea-1c70-4a7b-a71f-ca303da033c4 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:31.754181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:31.786940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772deff2-077f-433e-a4ca-ff9d0be394b5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:31.861937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cd4dd-ee5b-4a65-b559-433536c284f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:31.862773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cd4dd-ee5b-4a65-b559-433536c284f4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:31.863049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cd4dd-ee5b-4a65-b559-433536c284f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:31.863299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cd4dd-ee5b-4a65-b559-433536c284f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:31.882691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cd4dd-ee5b-4a65-b559-433536c284f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-514cd4dd-ee5b-4a65-b559-433536c284f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:31.891982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:31.894333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:31.896410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:31.896410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:31.896410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:31.896410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:31.896410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:31.896410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:31.896410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:31.896410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:31.896410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:31.896410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:31.896410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:31.896410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:31.900805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 186/354] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:31.904515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cd4dd-ee5b-4a65-b559-433536c284f4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:31.971638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed43ea-1c70-4a7b-a71f-ca303da033c4 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:31.985930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:32.049417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32380b62-a382-4c3b-9917-1a554090ab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:32.050717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32380b62-a382-4c3b-9917-1a554090ab93 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.051279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32380b62-a382-4c3b-9917-1a554090ab93 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.051560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32380b62-a382-4c3b-9917-1a554090ab93 None admin] PATH_INFO: `/v3/groups/f2c406148932437fa57d1eed86c91879` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.052859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32380b62-a382-4c3b-9917-1a554090ab93 None admin] RBAC: Authorizing `identity:get_group(group_id=f2c406148932437fa57d1eed86c91879)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:32.054387 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32380b62-a382-4c3b-9917-1a554090ab93 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:32.056330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 169/355] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:31 2022] GET /identity/v3/groups/f2c406148932437fa57d1eed86c91879 => generated 226 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:32.057184 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32380b62-a382-4c3b-9917-1a554090ab93 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:32.154312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cd4dd-ee5b-4a65-b559-433536c284f4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:32.212657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32380b62-a382-4c3b-9917-1a554090ab93 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:32.213515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:32.238485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399f236-5f97-4e0c-8e5c-c4976f243f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:32.239390 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f236-5f97-4e0c-8e5c-c4976f243f43 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.244653 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f236-5f97-4e0c-8e5c-c4976f243f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.245210 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f236-5f97-4e0c-8e5c-c4976f243f43 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.246359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399f236-5f97-4e0c-8e5c-c4976f243f43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:32.246938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399f236-5f97-4e0c-8e5c-c4976f243f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:32.250456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c399f236-5f97-4e0c-8e5c-c4976f243f43 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:32.257395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80f13e-e83c-487f-b230-64177c170768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:32.258105 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80f13e-e83c-487f-b230-64177c170768 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.258396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80f13e-e83c-487f-b230-64177c170768 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.258678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80f13e-e83c-487f-b230-64177c170768 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.259461 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80f13e-e83c-487f-b230-64177c170768 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:32.259772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80f13e-e83c-487f-b230-64177c170768 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:32.261210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80f13e-e83c-487f-b230-64177c170768 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:32.272222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 170/356] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:32.273166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80f13e-e83c-487f-b230-64177c170768 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:32.278669 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 187/357] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:32.289220 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4ac3f-5648-49da-a010-2c2bc130573b None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.289737 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4ac3f-5648-49da-a010-2c2bc130573b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.290240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4ac3f-5648-49da-a010-2c2bc130573b None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.293334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 188/358] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:32 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:32.295104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4ac3f-5648-49da-a010-2c2bc130573b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:32.315148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:32.394335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80f13e-e83c-487f-b230-64177c170768 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:32.436617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d011d7-f1a3-4b1c-a984-526457ee6654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:32.437824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d011d7-f1a3-4b1c-a984-526457ee6654 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.438176 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d011d7-f1a3-4b1c-a984-526457ee6654 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.438457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d011d7-f1a3-4b1c-a984-526457ee6654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.439376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d011d7-f1a3-4b1c-a984-526457ee6654 None admin] RBAC: Adding query filter params (group.id=f2c406148932437fa57d1eed86c91879, role.id=1b870be20b0c4159bfbbe73593799250, scope.project.id=bd637189b43c45c28839d066df463bc3) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:32.439699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d011d7-f1a3-4b1c-a984-526457ee6654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:32.441238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d011d7-f1a3-4b1c-a984-526457ee6654 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:32.444906 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4ac3f-5648-49da-a010-2c2bc130573b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:32.447606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 171/359] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:32 2022] GET /identity/v3/role_assignments?group.id=f2c406148932437fa57d1eed86c91879&scope.project.id=bd637189b43c45c28839d066df463bc3&role.id=1b870be20b0c4159bfbbe73593799250 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:32.448879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d011d7-f1a3-4b1c-a984-526457ee6654 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:32.488999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bef6d3-bbaf-4b3b-b5a4-01366e649d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:32.490364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bef6d3-bbaf-4b3b-b5a4-01366e649d24 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.491250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bef6d3-bbaf-4b3b-b5a4-01366e649d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.491757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bef6d3-bbaf-4b3b-b5a4-01366e649d24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.567878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:32.572355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d011d7-f1a3-4b1c-a984-526457ee6654 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:32.582955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bef6d3-bbaf-4b3b-b5a4-01366e649d24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:32.584926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bef6d3-bbaf-4b3b-b5a4-01366e649d24 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:32.589335 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:32.615978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee11735-69f1-4f7a-9ba8-6e06cdc63572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:32.616700 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11735-69f1-4f7a-9ba8-6e06cdc63572 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.617005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11735-69f1-4f7a-9ba8-6e06cdc63572 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.617360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11735-69f1-4f7a-9ba8-6e06cdc63572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.704591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee11735-69f1-4f7a-9ba8-6e06cdc63572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:32.706183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee11735-69f1-4f7a-9ba8-6e06cdc63572 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:32.708690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 189/360] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:32.710373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648c3a4-27f4-4136-a265-8598d72b938d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.710744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648c3a4-27f4-4136-a265-8598d72b938d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.711088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648c3a4-27f4-4136-a265-8598d72b938d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.731558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a648c3a4-27f4-4136-a265-8598d72b938d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:32.733123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a648c3a4-27f4-4136-a265-8598d72b938d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:32.822388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 190/361] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:32.823827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f69c70-5b9d-4416-9a73-8b24b9d23201 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.824122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f69c70-5b9d-4416-9a73-8b24b9d23201 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.824406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f69c70-5b9d-4416-9a73-8b24b9d23201 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.826246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 191/362] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:32.828304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f69c70-5b9d-4416-9a73-8b24b9d23201 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:32.851437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:32.873464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:32.880256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 172/363] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:32.881856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68f7f9-b85a-4480-9281-98909920e82d None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:32.882214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68f7f9-b85a-4480-9281-98909920e82d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:32.882565 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68f7f9-b85a-4480-9281-98909920e82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:32.901781 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-6c68f7f9-b85a-4480-9281-98909920e82d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:32.903098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6c68f7f9-b85a-4480-9281-98909920e82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:32.975774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f69c70-5b9d-4416-9a73-8b24b9d23201 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:32.991536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 173/364] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:32.992810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68f7f9-b85a-4480-9281-98909920e82d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.022759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc197dbe-8d28-4b62-ac23-e5e01b22a54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.024086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc197dbe-8d28-4b62-ac23-e5e01b22a54c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.024585 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc197dbe-8d28-4b62-ac23-e5e01b22a54c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.025041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc197dbe-8d28-4b62-ac23-e5e01b22a54c None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.026647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc197dbe-8d28-4b62-ac23-e5e01b22a54c None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:33.028435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc197dbe-8d28-4b62-ac23-e5e01b22a54c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:33.031340 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 192/365] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:32 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:33.032793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc197dbe-8d28-4b62-ac23-e5e01b22a54c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.144509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68f7f9-b85a-4480-9281-98909920e82d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.152285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:33.181044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc197dbe-8d28-4b62-ac23-e5e01b22a54c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.209003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:33.229956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f48b54-488b-4133-a4d3-8d428f264513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.230841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f48b54-488b-4133-a4d3-8d428f264513 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.231124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f48b54-488b-4133-a4d3-8d428f264513 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.231395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f48b54-488b-4133-a4d3-8d428f264513 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.232033 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01387aa1-3961-4116-bdf6-d3b001d61dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.232768 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f48b54-488b-4133-a4d3-8d428f264513 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:33.235240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f48b54-488b-4133-a4d3-8d428f264513 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:33.235982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01387aa1-3961-4116-bdf6-d3b001d61dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.236689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 193/366] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:33.237165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01387aa1-3961-4116-bdf6-d3b001d61dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.237482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f48b54-488b-4133-a4d3-8d428f264513 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.238358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01387aa1-3961-4116-bdf6-d3b001d61dc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.272655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01387aa1-3961-4116-bdf6-d3b001d61dc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-01387aa1-3961-4116-bdf6-d3b001d61dc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:33.293983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:33.297982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 174/367] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:33.299650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01387aa1-3961-4116-bdf6-d3b001d61dc9 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.383416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f48b54-488b-4133-a4d3-8d428f264513 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.423666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26293060-10c3-44c2-b410-bf4a16205908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.424393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26293060-10c3-44c2-b410-bf4a16205908 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.424718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26293060-10c3-44c2-b410-bf4a16205908 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.425004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26293060-10c3-44c2-b410-bf4a16205908 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.438015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26293060-10c3-44c2-b410-bf4a16205908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-26293060-10c3-44c2-b410-bf4a16205908 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:33.448849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:33.449801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:33.450595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:33.450595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:33.450595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:33.450595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:33.450595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:33.450595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:33.450595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:33.450595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:33.450595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:33.450595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:33.450595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:19:33.450595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:33.452849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 194/368] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:33.454403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26293060-10c3-44c2-b410-bf4a16205908 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.481064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:33.537481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01387aa1-3961-4116-bdf6-d3b001d61dc9 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.605613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26293060-10c3-44c2-b410-bf4a16205908 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.634379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0178c74-81cb-4480-ad51-3dba6e331297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.636240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0178c74-81cb-4480-ad51-3dba6e331297 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.636240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0178c74-81cb-4480-ad51-3dba6e331297 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.636240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0178c74-81cb-4480-ad51-3dba6e331297 None admin] PATH_INFO: `/v3/groups/f2c406148932437fa57d1eed86c91879` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.637473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0178c74-81cb-4480-ad51-3dba6e331297 None admin] RBAC: Authorizing `identity:get_group(group_id=f2c406148932437fa57d1eed86c91879)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:33.640693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0178c74-81cb-4480-ad51-3dba6e331297 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:33.643625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 175/369] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/groups/f2c406148932437fa57d1eed86c91879 => generated 226 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:33.643913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0178c74-81cb-4480-ad51-3dba6e331297 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.687863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafc2c6-d55b-4f1a-a1be-e0aed7c1760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.689022 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafc2c6-d55b-4f1a-a1be-e0aed7c1760f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.689447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafc2c6-d55b-4f1a-a1be-e0aed7c1760f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.689885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafc2c6-d55b-4f1a-a1be-e0aed7c1760f None admin] PATH_INFO: `/v3/groups/f2c406148932437fa57d1eed86c91879` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.691357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafc2c6-d55b-4f1a-a1be-e0aed7c1760f None admin] RBAC: Authorizing `identity:get_group(group_id=f2c406148932437fa57d1eed86c91879)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:33.694401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafc2c6-d55b-4f1a-a1be-e0aed7c1760f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:33.696431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 195/370] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/groups/f2c406148932437fa57d1eed86c91879 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:33.697521 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafc2c6-d55b-4f1a-a1be-e0aed7c1760f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.727349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:33.778997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0178c74-81cb-4480-ad51-3dba6e331297 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.807822 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:33.818569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d926daa-4288-448a-bed5-1eb65b455665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.819261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d926daa-4288-448a-bed5-1eb65b455665 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.819550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d926daa-4288-448a-bed5-1eb65b455665 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.819836 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d926daa-4288-448a-bed5-1eb65b455665 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.820677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d926daa-4288-448a-bed5-1eb65b455665 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:33.820992 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d926daa-4288-448a-bed5-1eb65b455665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:33.822392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d926daa-4288-448a-bed5-1eb65b455665 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:33.832293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 176/371] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:33.835230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d926daa-4288-448a-bed5-1eb65b455665 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:33.838515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafc2c6-d55b-4f1a-a1be-e0aed7c1760f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:33.887320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0837f2-5fce-4b20-af55-03f567544ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:33.888427 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0837f2-5fce-4b20-af55-03f567544ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:33.888984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0837f2-5fce-4b20-af55-03f567544ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:33.889460 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0837f2-5fce-4b20-af55-03f567544ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:33.891108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0837f2-5fce-4b20-af55-03f567544ff0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:33.891601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0837f2-5fce-4b20-af55-03f567544ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:33.893399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0837f2-5fce-4b20-af55-03f567544ff0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:33.904163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 196/372] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:33.905935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0837f2-5fce-4b20-af55-03f567544ff0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:34.001044 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d926daa-4288-448a-bed5-1eb65b455665 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.052650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:34.076677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0837f2-5fce-4b20-af55-03f567544ff0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.078715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd0bad-6eee-44d0-9cc3-d1581d62f95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:34.080375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd0bad-6eee-44d0-9cc3-d1581d62f95e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.080952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd0bad-6eee-44d0-9cc3-d1581d62f95e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.082998 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd0bad-6eee-44d0-9cc3-d1581d62f95e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.085519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd0bad-6eee-44d0-9cc3-d1581d62f95e None admin] RBAC: Adding query filter params (group.id=f2c406148932437fa57d1eed86c91879, role.id=80eea410deca46e2924f27afe5136d55, scope.project.id=8f4b4898eca3450daecc84800ff218c9) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:34.086105 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd0bad-6eee-44d0-9cc3-d1581d62f95e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:34.089995 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd0bad-6eee-44d0-9cc3-d1581d62f95e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:34.106091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 177/373] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/role_assignments?group.id=f2c406148932437fa57d1eed86c91879&scope.project.id=8f4b4898eca3450daecc84800ff218c9&role.id=80eea410deca46e2924f27afe5136d55 => generated 260 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:34.109326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd0bad-6eee-44d0-9cc3-d1581d62f95e None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:34.164623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29556ff6-39b3-4c5c-86fa-d04d48071227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:34.165792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29556ff6-39b3-4c5c-86fa-d04d48071227 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.166318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29556ff6-39b3-4c5c-86fa-d04d48071227 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.166837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29556ff6-39b3-4c5c-86fa-d04d48071227 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.168549 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29556ff6-39b3-4c5c-86fa-d04d48071227 None admin] RBAC: Adding query filter params (group.id=f2c406148932437fa57d1eed86c91879, role.id=1b870be20b0c4159bfbbe73593799250, scope.project.id=8f4b4898eca3450daecc84800ff218c9) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:34.169130 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29556ff6-39b3-4c5c-86fa-d04d48071227 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:34.172849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29556ff6-39b3-4c5c-86fa-d04d48071227 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:34.184861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 197/374] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:33 2022] GET /identity/v3/role_assignments?group.id=f2c406148932437fa57d1eed86c91879&scope.project.id=8f4b4898eca3450daecc84800ff218c9&role.id=1b870be20b0c4159bfbbe73593799250 => generated 260 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:34.190202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29556ff6-39b3-4c5c-86fa-d04d48071227 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:34.208513 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:34.398761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd0bad-6eee-44d0-9cc3-d1581d62f95e None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.404376 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29556ff6-39b3-4c5c-86fa-d04d48071227 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.470516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d7b60-63af-4799-9b7b-8000f93dc34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:34.471443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d7b60-63af-4799-9b7b-8000f93dc34c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.471728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d7b60-63af-4799-9b7b-8000f93dc34c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.472005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d7b60-63af-4799-9b7b-8000f93dc34c None admin] PATH_INFO: `/v3/users/ee4895e470d54a038729ec1c12b7cd22` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.473535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d7b60-63af-4799-9b7b-8000f93dc34c None admin] RBAC: Authorizing `identity:get_user(user_id=ee4895e470d54a038729ec1c12b7cd22)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:34.475574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d7b60-63af-4799-9b7b-8000f93dc34c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:34.478090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 198/375] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:19:34 2022] GET /identity/v3/users/ee4895e470d54a038729ec1c12b7cd22 => generated 293 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:34.479293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8a5d5-fd60-4e09-b7a4-6a8a8a5e57c4 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.479572 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8a5d5-fd60-4e09-b7a4-6a8a8a5e57c4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.479853 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8a5d5-fd60-4e09-b7a4-6a8a8a5e57c4 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.481424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 199/376] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:34.481752 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03580d-3ba5-4c78-a593-3bc712d06aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:34.482430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03580d-3ba5-4c78-a593-3bc712d06aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.482780 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03580d-3ba5-4c78-a593-3bc712d06aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.483170 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03580d-3ba5-4c78-a593-3bc712d06aa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.484793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8a5d5-fd60-4e09-b7a4-6a8a8a5e57c4 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:34.561350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:34.603832 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:34.608113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03580d-3ba5-4c78-a593-3bc712d06aa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:34.610476 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03580d-3ba5-4c78-a593-3bc712d06aa7 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:34.685674 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8a5d5-fd60-4e09-b7a4-6a8a8a5e57c4 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:34.790262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711b395-81c3-4b82-bdd1-f7df48a753d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:34.791584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711b395-81c3-4b82-bdd1-f7df48a753d1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.792068 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711b395-81c3-4b82-bdd1-f7df48a753d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.792562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711b395-81c3-4b82-bdd1-f7df48a753d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.793633 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711b395-81c3-4b82-bdd1-f7df48a753d1 None admin] RBAC: Adding query filter params (role.id=35dbb86a4b774e0a9d862b6fa21cd947, user.id=ee4895e470d54a038729ec1c12b7cd22, scope.system=all) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:34.794139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711b395-81c3-4b82-bdd1-f7df48a753d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:34.795936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711b395-81c3-4b82-bdd1-f7df48a753d1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:34.821043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 200/377] 10.222.0.42 () {56 vars in 1308 bytes} [Thu Jul 21 22:19:34 2022] GET /identity/v3/role_assignments?user.id=ee4895e470d54a038729ec1c12b7cd22&scope.system=all&role.id=35dbb86a4b774e0a9d862b6fa21cd947 => generated 510 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:34.830074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9974f-9d7d-4cb3-b3dc-cf6d149b96b6 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:34.830738 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9974f-9d7d-4cb3-b3dc-cf6d149b96b6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:34.831212 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9974f-9d7d-4cb3-b3dc-cf6d149b96b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:34.858887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 178/378] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 752 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:34.864757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03580d-3ba5-4c78-a593-3bc712d06aa7 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:34.874952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-fdc9974f-9d7d-4cb3-b3dc-cf6d149b96b6 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:34.877517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fdc9974f-9d7d-4cb3-b3dc-cf6d149b96b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:35.001036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:35.029167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 201/379] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:35.033221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03580d-3ba5-4c78-a593-3bc712d06aa7 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:35.063217 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9974f-9d7d-4cb3-b3dc-cf6d149b96b6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.127602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0ac75-a512-4b0b-913f-3bafeaf289c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:35.128517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0ac75-a512-4b0b-913f-3bafeaf289c1 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.128815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0ac75-a512-4b0b-913f-3bafeaf289c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.129092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0ac75-a512-4b0b-913f-3bafeaf289c1 None admin] PATH_INFO: `/v3/projects/5705979642f541aa9229c56505a6ee6e` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.133446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a0ac75-a512-4b0b-913f-3bafeaf289c1 None admin] RBAC: Authorizing `identity:get_project(project_id=5705979642f541aa9229c56505a6ee6e)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:35.134963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a0ac75-a512-4b0b-913f-3bafeaf289c1 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:35.137264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 179/380] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:34 2022] GET /identity/v3/projects/5705979642f541aa9229c56505a6ee6e => generated 344 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:35.143099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0ac75-a512-4b0b-913f-3bafeaf289c1 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.178097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:35.231486 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9974f-9d7d-4cb3-b3dc-cf6d149b96b6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:35.275929 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b8f95-1a25-446a-a7ed-a19a0e21c342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:35.277274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b8f95-1a25-446a-a7ed-a19a0e21c342 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.277889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b8f95-1a25-446a-a7ed-a19a0e21c342 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.278394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b8f95-1a25-446a-a7ed-a19a0e21c342 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.291932 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b8f95-1a25-446a-a7ed-a19a0e21c342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b15b8f95-1a25-446a-a7ed-a19a0e21c342 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:35.301314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:35.302298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:35.303157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:35.303157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:35.303157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:35.303157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:35.303157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:35.303157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:35.303157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:35.303157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:35.303157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:35.303157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:35.303157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:35.303157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:35.306280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 202/381] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:35.315612 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b8f95-1a25-446a-a7ed-a19a0e21c342 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.431056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0ac75-a512-4b0b-913f-3bafeaf289c1 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:35.489933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b9856-250e-4657-ae32-f24c65284216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:35.490901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b9856-250e-4657-ae32-f24c65284216 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.491413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b9856-250e-4657-ae32-f24c65284216 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.491869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b8f95-1a25-446a-a7ed-a19a0e21c342 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:35.491977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b9856-250e-4657-ae32-f24c65284216 None admin] PATH_INFO: `/v3/groups/2c680d1ff0f34e46a520cb30aa1ddfc6` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.513239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b9856-250e-4657-ae32-f24c65284216 None admin] RBAC: Authorizing `identity:get_group(group_id=2c680d1ff0f34e46a520cb30aa1ddfc6)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:35.515425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b9856-250e-4657-ae32-f24c65284216 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:35.517998 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 180/382] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/groups/2c680d1ff0f34e46a520cb30aa1ddfc6 => generated 229 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:35.519656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bde7b-7273-493f-afbe-7b9c81bd3d3e None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.520242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bde7b-7273-493f-afbe-7b9c81bd3d3e None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.520778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bde7b-7273-493f-afbe-7b9c81bd3d3e None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.522945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 181/383] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:35.523989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bde7b-7273-493f-afbe-7b9c81bd3d3e None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.537650 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:35.553253 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:35.567919 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1010674-db2b-4fc0-a564-c1bb1540ee2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:35.569389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1010674-db2b-4fc0-a564-c1bb1540ee2c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.569389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1010674-db2b-4fc0-a564-c1bb1540ee2c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.569754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1010674-db2b-4fc0-a564-c1bb1540ee2c None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.571199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1010674-db2b-4fc0-a564-c1bb1540ee2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:35.571761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1010674-db2b-4fc0-a564-c1bb1540ee2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:35.574163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1010674-db2b-4fc0-a564-c1bb1540ee2c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:35.589537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 203/384] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:35.591942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e258d23-221d-4f3a-b21e-9c27b0852168 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.593510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e258d23-221d-4f3a-b21e-9c27b0852168 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.594383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e258d23-221d-4f3a-b21e-9c27b0852168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.620853 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0e258d23-221d-4f3a-b21e-9c27b0852168 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:35.623720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0e258d23-221d-4f3a-b21e-9c27b0852168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:35.690570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bde7b-7273-493f-afbe-7b9c81bd3d3e None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:35.735187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3b1b5-c9a9-4de4-9d88-077786882a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:35.736269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3b1b5-c9a9-4de4-9d88-077786882a22 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.736812 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3b1b5-c9a9-4de4-9d88-077786882a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.738266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3b1b5-c9a9-4de4-9d88-077786882a22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.739444 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3b1b5-c9a9-4de4-9d88-077786882a22 None admin] RBAC: Adding query filter params (group.id=2c680d1ff0f34e46a520cb30aa1ddfc6, role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=5705979642f541aa9229c56505a6ee6e) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:35.739965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3b1b5-c9a9-4de4-9d88-077786882a22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:35.741682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3b1b5-c9a9-4de4-9d88-077786882a22 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:35.749473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 182/385] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/role_assignments?group.id=2c680d1ff0f34e46a520cb30aa1ddfc6&scope.project.id=5705979642f541aa9229c56505a6ee6e&role.id=f957e09d047b47cfa0891d25b4181263 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:35.750723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3b1b5-c9a9-4de4-9d88-077786882a22 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.766782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 204/386] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:35.794208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e258d23-221d-4f3a-b21e-9c27b0852168 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:35.884153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:35.919493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3b1b5-c9a9-4de4-9d88-077786882a22 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:35.982401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8cfc2-a1bb-471a-8478-c78779e5970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:35.983822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8cfc2-a1bb-471a-8478-c78779e5970a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:35.984121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8cfc2-a1bb-471a-8478-c78779e5970a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:35.984397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8cfc2-a1bb-471a-8478-c78779e5970a None admin] PATH_INFO: `/v3/groups/f2c406148932437fa57d1eed86c91879` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:35.985731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8cfc2-a1bb-471a-8478-c78779e5970a None admin] RBAC: Authorizing `identity:get_group(group_id=f2c406148932437fa57d1eed86c91879)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:35.987282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8cfc2-a1bb-471a-8478-c78779e5970a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:35.989165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 183/387] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/groups/f2c406148932437fa57d1eed86c91879 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:35.995191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8cfc2-a1bb-471a-8478-c78779e5970a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:36.045061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e258d23-221d-4f3a-b21e-9c27b0852168 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:36.079830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:36.137230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb306c8-9ad5-4d6b-9399-5aa44a2a9b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:36.138673 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb306c8-9ad5-4d6b-9399-5aa44a2a9b1a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.139179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb306c8-9ad5-4d6b-9399-5aa44a2a9b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.141288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb306c8-9ad5-4d6b-9399-5aa44a2a9b1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.174365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb306c8-9ad5-4d6b-9399-5aa44a2a9b1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-acb306c8-9ad5-4d6b-9399-5aa44a2a9b1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:36.191622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:36.192695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:36.193498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:36.193498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:36.193498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:36.193498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:36.193498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:36.193498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:36.193498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:36.193498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:36.193498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:36.193498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:36.193498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:36.193498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:36.196517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 205/388] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:36.200761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb306c8-9ad5-4d6b-9399-5aa44a2a9b1a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:36.225661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8cfc2-a1bb-471a-8478-c78779e5970a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:36.321522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a468f-8bb2-493e-a967-b01ec6114ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:36.322347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a468f-8bb2-493e-a967-b01ec6114ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.322638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a468f-8bb2-493e-a967-b01ec6114ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.322921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a468f-8bb2-493e-a967-b01ec6114ff4 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.324173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a468f-8bb2-493e-a967-b01ec6114ff4 None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:36.325676 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a468f-8bb2-493e-a967-b01ec6114ff4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:36.332760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 184/389] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:35 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:36.340438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a468f-8bb2-493e-a967-b01ec6114ff4 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:36.386499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:36.462007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb306c8-9ad5-4d6b-9399-5aa44a2a9b1a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:36.541222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5dc11-70c8-47de-b23e-7d7f52a7a91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:36.542723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5dc11-70c8-47de-b23e-7d7f52a7a91f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.543436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5dc11-70c8-47de-b23e-7d7f52a7a91f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.544132 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5dc11-70c8-47de-b23e-7d7f52a7a91f None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.546726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5dc11-70c8-47de-b23e-7d7f52a7a91f None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:36.547504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5dc11-70c8-47de-b23e-7d7f52a7a91f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:36.550377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5dc11-70c8-47de-b23e-7d7f52a7a91f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:36.570957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 206/390] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:36.579569 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5dc11-70c8-47de-b23e-7d7f52a7a91f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:36.658771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a468f-8bb2-493e-a967-b01ec6114ff4 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:36.730630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:36.754955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53571129-c455-4e91-b48b-8a3039d74277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:36.755875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53571129-c455-4e91-b48b-8a3039d74277 None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.756172 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53571129-c455-4e91-b48b-8a3039d74277 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.756461 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53571129-c455-4e91-b48b-8a3039d74277 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3/groups/f2c406148932437fa57d1eed86c91879/roles/80eea410deca46e2924f27afe5136d55` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.764156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53571129-c455-4e91-b48b-8a3039d74277 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd637189b43c45c28839d066df463bc3, group_id=f2c406148932437fa57d1eed86c91879, role_id=80eea410deca46e2924f27afe5136d55)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:36.766267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53571129-c455-4e91-b48b-8a3039d74277 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:36.803731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 185/391] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jul 21 22:19:36 2022] PUT /identity/v3/projects/bd637189b43c45c28839d066df463bc3/groups/f2c406148932437fa57d1eed86c91879/roles/80eea410deca46e2924f27afe5136d55 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:36.873041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5dc11-70c8-47de-b23e-7d7f52a7a91f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:36.939541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2377e-8ab7-4b42-a6d2-beb16759ec8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:36.941020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2377e-8ab7-4b42-a6d2-beb16759ec8f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:36.941725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2377e-8ab7-4b42-a6d2-beb16759ec8f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:36.942342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2377e-8ab7-4b42-a6d2-beb16759ec8f None admin] PATH_INFO: `/v3/groups/f2c406148932437fa57d1eed86c91879` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:36.945046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2377e-8ab7-4b42-a6d2-beb16759ec8f None admin] RBAC: Authorizing `identity:get_group(group_id=f2c406148932437fa57d1eed86c91879)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:36.948335 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2377e-8ab7-4b42-a6d2-beb16759ec8f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:36.955896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 207/392] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:36 2022] GET /identity/v3/groups/f2c406148932437fa57d1eed86c91879 => generated 226 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:36.965703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53571129-c455-4e91-b48b-8a3039d74277 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:37.109360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:37.145216 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53571129-c455-4e91-b48b-8a3039d74277 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:37.185851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a9f78-aa44-4936-889e-b6d48f88fa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:37.187816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a9f78-aa44-4936-889e-b6d48f88fa53 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.188309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a9f78-aa44-4936-889e-b6d48f88fa53 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.188810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a9f78-aa44-4936-889e-b6d48f88fa53 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.190334 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a9f78-aa44-4936-889e-b6d48f88fa53 None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:37.192056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a9f78-aa44-4936-889e-b6d48f88fa53 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:37.194512 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 186/393] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:36 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:37.200162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2377e-8ab7-4b42-a6d2-beb16759ec8f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:37.355941 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2377e-8ab7-4b42-a6d2-beb16759ec8f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:37.359632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:37.405604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c29866-61f3-474b-8da7-8f52383776b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:37.406687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c29866-61f3-474b-8da7-8f52383776b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.406687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c29866-61f3-474b-8da7-8f52383776b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.406996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c29866-61f3-474b-8da7-8f52383776b3 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3/groups/f2c406148932437fa57d1eed86c91879/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.417305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c39020-17f8-4e71-b79f-aa3d3510fc56 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.418239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c39020-17f8-4e71-b79f-aa3d3510fc56 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.419118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c39020-17f8-4e71-b79f-aa3d3510fc56 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.419388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c29866-61f3-474b-8da7-8f52383776b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd637189b43c45c28839d066df463bc3, group_id=f2c406148932437fa57d1eed86c91879, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:37.421491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c29866-61f3-474b-8da7-8f52383776b3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:37.429082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 187/394] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:37.435757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97714254-85b7-436f-a98b-5333d0eefcbf None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.438473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97714254-85b7-436f-a98b-5333d0eefcbf None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.439312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97714254-85b7-436f-a98b-5333d0eefcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.453425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 208/395] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jul 21 22:19:37 2022] PUT /identity/v3/projects/bd637189b43c45c28839d066df463bc3/groups/f2c406148932437fa57d1eed86c91879/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:19:37.477464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-97714254-85b7-436f-a98b-5333d0eefcbf None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:37.479920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-97714254-85b7-436f-a98b-5333d0eefcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:37.610491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334bebb-1fef-4fee-b178-c405ae1abd9d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.610491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334bebb-1fef-4fee-b178-c405ae1abd9d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.611431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334bebb-1fef-4fee-b178-c405ae1abd9d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.615488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 209/396] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:37.618053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e57763-e140-4c2e-a8c7-84673740da23 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.618394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e57763-e140-4c2e-a8c7-84673740da23 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.618652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e57763-e140-4c2e-a8c7-84673740da23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.646482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 188/397] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:37.669171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-08e57763-e140-4c2e-a8c7-84673740da23 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:37.671620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-08e57763-e140-4c2e-a8c7-84673740da23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:37.685749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97714254-85b7-436f-a98b-5333d0eefcbf None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:37.787544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 210/398] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:37.812786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e57763-e140-4c2e-a8c7-84673740da23 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:37.841144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:37.897392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97714254-85b7-436f-a98b-5333d0eefcbf None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:37.959110 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c83da-9ed5-4a35-8448-4d708985f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:37.960362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c83da-9ed5-4a35-8448-4d708985f183 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:37.960927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c83da-9ed5-4a35-8448-4d708985f183 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:37.961390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c83da-9ed5-4a35-8448-4d708985f183 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:37.999655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c83da-9ed5-4a35-8448-4d708985f183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-3c6c83da-9ed5-4a35-8448-4d708985f183 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:38.014852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:38.015818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:38.016644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:38.016644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:38.016644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:38.016644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:38.016644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:38.016644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:38.016644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:38.016644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:38.016644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:38.016644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:38.016644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:38.016644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:38.017817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 189/399] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:38.026211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c83da-9ed5-4a35-8448-4d708985f183 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.104873 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:38.114051 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e57763-e140-4c2e-a8c7-84673740da23 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:38.175864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4abee-96ed-4dae-9e7c-4646028cd5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:38.176734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4abee-96ed-4dae-9e7c-4646028cd5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.177018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4abee-96ed-4dae-9e7c-4646028cd5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.177292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4abee-96ed-4dae-9e7c-4646028cd5d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.203444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4abee-96ed-4dae-9e7c-4646028cd5d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-69f4abee-96ed-4dae-9e7c-4646028cd5d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:38.220451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:38.221455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:38.222275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:38.222275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:38.222275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:38.222275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:38.222275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:38.222275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:38.222275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:38.222275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:38.222275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:38.222275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:38.222275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:38.222275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:38.222976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 211/400] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:38.229068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c83da-9ed5-4a35-8448-4d708985f183 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:38.231593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4abee-96ed-4dae-9e7c-4646028cd5d1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.307255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:38.314677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec76a8-45fa-4fc0-9b45-9985dd62d4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:38.315371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec76a8-45fa-4fc0-9b45-9985dd62d4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.315664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec76a8-45fa-4fc0-9b45-9985dd62d4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.315948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec76a8-45fa-4fc0-9b45-9985dd62d4e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.316749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec76a8-45fa-4fc0-9b45-9985dd62d4e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:38.317067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec76a8-45fa-4fc0-9b45-9985dd62d4e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:38.318482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec76a8-45fa-4fc0-9b45-9985dd62d4e6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:38.346409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 190/401] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:38.350004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec76a8-45fa-4fc0-9b45-9985dd62d4e6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.442266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4abee-96ed-4dae-9e7c-4646028cd5d1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:38.517464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82810f8-9bec-43cf-ad55-44c449383677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:38.518072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82810f8-9bec-43cf-ad55-44c449383677 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.518361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82810f8-9bec-43cf-ad55-44c449383677 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.518736 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82810f8-9bec-43cf-ad55-44c449383677 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.519574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82810f8-9bec-43cf-ad55-44c449383677 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:38.519888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82810f8-9bec-43cf-ad55-44c449383677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:38.521326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82810f8-9bec-43cf-ad55-44c449383677 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:38.542127 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 212/402] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:38.547008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82810f8-9bec-43cf-ad55-44c449383677 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.552944 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec76a8-45fa-4fc0-9b45-9985dd62d4e6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:38.587007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:38.629840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b10f9b-4ba4-4b71-9d9b-099bfb482956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:38.630727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b10f9b-4ba4-4b71-9d9b-099bfb482956 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.631013 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b10f9b-4ba4-4b71-9d9b-099bfb482956 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.631293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b10f9b-4ba4-4b71-9d9b-099bfb482956 None admin] PATH_INFO: `/v3/groups/f2c406148932437fa57d1eed86c91879` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.634592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b10f9b-4ba4-4b71-9d9b-099bfb482956 None admin] RBAC: Authorizing `identity:get_group(group_id=f2c406148932437fa57d1eed86c91879)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:38.636084 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b10f9b-4ba4-4b71-9d9b-099bfb482956 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:38.637842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 191/403] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/groups/f2c406148932437fa57d1eed86c91879 => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:38.646106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b10f9b-4ba4-4b71-9d9b-099bfb482956 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.812243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82810f8-9bec-43cf-ad55-44c449383677 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:38.824971 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:38.925872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299266f8-9908-410f-b841-c8288085ff51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:38.926944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299266f8-9908-410f-b841-c8288085ff51 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:38.927331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299266f8-9908-410f-b841-c8288085ff51 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:38.927718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299266f8-9908-410f-b841-c8288085ff51 None admin] PATH_INFO: `/v3/groups/f2c406148932437fa57d1eed86c91879` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:38.929332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299266f8-9908-410f-b841-c8288085ff51 None admin] RBAC: Authorizing `identity:get_group(group_id=f2c406148932437fa57d1eed86c91879)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:38.931165 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299266f8-9908-410f-b841-c8288085ff51 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:38.933842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 213/404] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/groups/f2c406148932437fa57d1eed86c91879 => generated 226 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:38.940285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299266f8-9908-410f-b841-c8288085ff51 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:38.955705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b10f9b-4ba4-4b71-9d9b-099bfb482956 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:39.055454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf6c90-7e95-415b-bb34-9041cf126d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:39.056269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf6c90-7e95-415b-bb34-9041cf126d71 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.056694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf6c90-7e95-415b-bb34-9041cf126d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.057080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf6c90-7e95-415b-bb34-9041cf126d71 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.058547 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf6c90-7e95-415b-bb34-9041cf126d71 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:39.060787 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf6c90-7e95-415b-bb34-9041cf126d71 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:39.063155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 192/405] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:39.071245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae17fd-9e0d-41b8-b2a8-4eebb49915b1 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.071245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae17fd-9e0d-41b8-b2a8-4eebb49915b1 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.071245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae17fd-9e0d-41b8-b2a8-4eebb49915b1 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.071245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 193/406] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:39.072979 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fe679-761b-470f-a02e-b221d1304aac None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.073380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fe679-761b-470f-a02e-b221d1304aac None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.073766 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fe679-761b-470f-a02e-b221d1304aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.093686 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-787fe679-761b-470f-a02e-b221d1304aac None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:39.095692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-787fe679-761b-470f-a02e-b221d1304aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:39.156906 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:39.192702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299266f8-9908-410f-b841-c8288085ff51 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:39.200880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 194/407] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:39.202121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fe679-761b-470f-a02e-b221d1304aac None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:39.249023 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10119a7d-219b-47ee-9133-6a92d89a09b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:39.250461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10119a7d-219b-47ee-9133-6a92d89a09b2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.250964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10119a7d-219b-47ee-9133-6a92d89a09b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.251338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10119a7d-219b-47ee-9133-6a92d89a09b2 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.253044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10119a7d-219b-47ee-9133-6a92d89a09b2 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:39.255570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10119a7d-219b-47ee-9133-6a92d89a09b2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:39.257775 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 214/408] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:38 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:39.259683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10119a7d-219b-47ee-9133-6a92d89a09b2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:39.320553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:39.384907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fe679-761b-470f-a02e-b221d1304aac None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:39.468312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10119a7d-219b-47ee-9133-6a92d89a09b2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:39.470610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c67402-2b8d-4226-a628-5049171812b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:39.472670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c67402-2b8d-4226-a628-5049171812b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.473524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c67402-2b8d-4226-a628-5049171812b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.474319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c67402-2b8d-4226-a628-5049171812b2 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9/groups/f2c406148932437fa57d1eed86c91879/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.478029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c67402-2b8d-4226-a628-5049171812b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f4b4898eca3450daecc84800ff218c9, group_id=f2c406148932437fa57d1eed86c91879, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:39.481325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c67402-2b8d-4226-a628-5049171812b2 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:39.519098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 195/409] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jul 21 22:19:39 2022] PUT /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9/groups/f2c406148932437fa57d1eed86c91879/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:39.519098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c67402-2b8d-4226-a628-5049171812b2 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:39.563562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc723a-d73b-47d5-8137-e8c1b77fa951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:39.565941 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc723a-d73b-47d5-8137-e8c1b77fa951 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.566287 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc723a-d73b-47d5-8137-e8c1b77fa951 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.566574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc723a-d73b-47d5-8137-e8c1b77fa951 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.607354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc723a-d73b-47d5-8137-e8c1b77fa951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-3ffc723a-d73b-47d5-8137-e8c1b77fa951 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:39.634974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:39.636780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:39.638208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 215/410] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:39.654015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4412a4a-9ce1-44d0-a357-b9d15ba9f921 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.654452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4412a4a-9ce1-44d0-a357-b9d15ba9f921 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.654685 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4412a4a-9ce1-44d0-a357-b9d15ba9f921 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.661510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 216/411] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:19:39.663070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ff568-737e-4ac2-bd74-9459ce435762 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.663367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ff568-737e-4ac2-bd74-9459ce435762 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.663647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ff568-737e-4ac2-bd74-9459ce435762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.683905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e41ff568-737e-4ac2-bd74-9459ce435762 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:39.685176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e41ff568-737e-4ac2-bd74-9459ce435762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:39.695749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:39.755642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c67402-2b8d-4226-a628-5049171812b2 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:39.785836 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:39.813222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 217/412] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:39.814642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ff568-737e-4ac2-bd74-9459ce435762 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:39.842015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e34795-3064-47bc-a06b-3a19ea708ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:39.843710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e34795-3064-47bc-a06b-3a19ea708ac3 None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:39.844225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e34795-3064-47bc-a06b-3a19ea708ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:39.844766 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e34795-3064-47bc-a06b-3a19ea708ac3 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9/groups/f2c406148932437fa57d1eed86c91879/roles/80eea410deca46e2924f27afe5136d55` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:39.854902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e34795-3064-47bc-a06b-3a19ea708ac3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f4b4898eca3450daecc84800ff218c9, group_id=f2c406148932437fa57d1eed86c91879, role_id=80eea410deca46e2924f27afe5136d55)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:39.858396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e34795-3064-47bc-a06b-3a19ea708ac3 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:39.893625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 196/413] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jul 21 22:19:39 2022] PUT /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9/groups/f2c406148932437fa57d1eed86c91879/roles/80eea410deca46e2924f27afe5136d55 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:39.895430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e34795-3064-47bc-a06b-3a19ea708ac3 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.015122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ff568-737e-4ac2-bd74-9459ce435762 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.102117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867dd66-b62e-4199-b660-7e8e5e967560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.102973 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867dd66-b62e-4199-b660-7e8e5e967560 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.104552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867dd66-b62e-4199-b660-7e8e5e967560 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.105057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867dd66-b62e-4199-b660-7e8e5e967560 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.106831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867dd66-b62e-4199-b660-7e8e5e967560 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:40.107593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867dd66-b62e-4199-b660-7e8e5e967560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:40.111191 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867dd66-b62e-4199-b660-7e8e5e967560 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:40.127064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 218/414] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:40.128909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ba083-f6ad-45c4-a33d-eef61b8cc06c None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.129420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ba083-f6ad-45c4-a33d-eef61b8cc06c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.129922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ba083-f6ad-45c4-a33d-eef61b8cc06c None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.132543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 219/415] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:40.134716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ba083-f6ad-45c4-a33d-eef61b8cc06c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.135093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e34795-3064-47bc-a06b-3a19ea708ac3 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.141900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:40.199518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082d6ac-b7cf-40e7-8c61-7402a2e0c188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.200770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082d6ac-b7cf-40e7-8c61-7402a2e0c188 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.201296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082d6ac-b7cf-40e7-8c61-7402a2e0c188 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.201845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082d6ac-b7cf-40e7-8c61-7402a2e0c188 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.218668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082d6ac-b7cf-40e7-8c61-7402a2e0c188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-9082d6ac-b7cf-40e7-8c61-7402a2e0c188 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:40.239881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:40.240984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:40.242500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 197/416] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:40.253191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2dad2-7351-40c1-a284-eed25143921c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.253191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2dad2-7351-40c1-a284-eed25143921c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.253191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2dad2-7351-40c1-a284-eed25143921c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.290968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-33b2dad2-7351-40c1-a284-eed25143921c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:40.293080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-33b2dad2-7351-40c1-a284-eed25143921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:40.332027 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ba083-f6ad-45c4-a33d-eef61b8cc06c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.361066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:40.375981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de27543-893d-439e-899e-bb51886ecdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.377324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de27543-893d-439e-899e-bb51886ecdda None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.377808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de27543-893d-439e-899e-bb51886ecdda None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.378275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de27543-893d-439e-899e-bb51886ecdda None admin] PATH_INFO: `/v3/groups/2c680d1ff0f34e46a520cb30aa1ddfc6` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.379772 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de27543-893d-439e-899e-bb51886ecdda None admin] RBAC: Authorizing `identity:get_group(group_id=2c680d1ff0f34e46a520cb30aa1ddfc6)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:40.381505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de27543-893d-439e-899e-bb51886ecdda None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:40.383804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 220/417] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:40 2022] GET /identity/v3/groups/2c680d1ff0f34e46a520cb30aa1ddfc6 => generated 229 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:40.384966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de27543-893d-439e-899e-bb51886ecdda None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.434213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 198/418] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:40.434213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2dad2-7351-40c1-a284-eed25143921c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.493525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:40.533882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de27543-893d-439e-899e-bb51886ecdda None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.573247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64081553-1c70-4962-8900-d324d86d7449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.574295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64081553-1c70-4962-8900-d324d86d7449 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.574797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64081553-1c70-4962-8900-d324d86d7449 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.575263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64081553-1c70-4962-8900-d324d86d7449 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.576253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64081553-1c70-4962-8900-d324d86d7449 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:40.576859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64081553-1c70-4962-8900-d324d86d7449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:40.578606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64081553-1c70-4962-8900-d324d86d7449 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:40.592909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 221/419] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:40.593093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2dad2-7351-40c1-a284-eed25143921c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.594304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64081553-1c70-4962-8900-d324d86d7449 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.636351 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395326d-42d7-49a8-a707-a78e0366a35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.637148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395326d-42d7-49a8-a707-a78e0366a35d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.637432 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395326d-42d7-49a8-a707-a78e0366a35d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.637706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395326d-42d7-49a8-a707-a78e0366a35d None admin] PATH_INFO: `/v3/projects/5705979642f541aa9229c56505a6ee6e` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.638984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395326d-42d7-49a8-a707-a78e0366a35d None admin] RBAC: Authorizing `identity:get_project(project_id=5705979642f541aa9229c56505a6ee6e)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:40.641380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395326d-42d7-49a8-a707-a78e0366a35d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:40.643459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 199/420] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:40 2022] GET /identity/v3/projects/5705979642f541aa9229c56505a6ee6e => generated 344 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:40.644265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395326d-42d7-49a8-a707-a78e0366a35d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.689697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:40.756755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395326d-42d7-49a8-a707-a78e0366a35d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.777661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64081553-1c70-4962-8900-d324d86d7449 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:40.799054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec00d5-1769-47cd-9ea3-9878aa9749e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.800015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec00d5-1769-47cd-9ea3-9878aa9749e2 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.800447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec00d5-1769-47cd-9ea3-9878aa9749e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.800928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec00d5-1769-47cd-9ea3-9878aa9749e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.855055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1b020-a8a9-4847-92e8-67b0760fe015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:40.856634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1b020-a8a9-4847-92e8-67b0760fe015 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:40.857278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1b020-a8a9-4847-92e8-67b0760fe015 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:40.858223 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1b020-a8a9-4847-92e8-67b0760fe015 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:40.862307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:40.885767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec00d5-1769-47cd-9ea3-9878aa9749e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:40.886309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1b020-a8a9-4847-92e8-67b0760fe015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:40.888134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec00d5-1769-47cd-9ea3-9878aa9749e2 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-75f1b020-a8a9-4847-92e8-67b0760fe015 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:40.903066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:40.904064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:40.905517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:40.905517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:40.905517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:40.905517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:40.905517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:40.905517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:40.905517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:40.905517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:40.905517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:40.905517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:40.905517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:40.905517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:40.913084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 222/421] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:40.914681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1b020-a8a9-4847-92e8-67b0760fe015 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:40.937912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:41.009289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 200/422] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:41.015042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec00d5-1769-47cd-9ea3-9878aa9749e2 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:41.044926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1b020-a8a9-4847-92e8-67b0760fe015 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.088119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebd5ea-6909-4068-b09a-d650ec3dc3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.089245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebd5ea-6909-4068-b09a-d650ec3dc3b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.090119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebd5ea-6909-4068-b09a-d650ec3dc3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.090119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebd5ea-6909-4068-b09a-d650ec3dc3b7 None admin] PATH_INFO: `/v3/projects/5705979642f541aa9229c56505a6ee6e/groups/2c680d1ff0f34e46a520cb30aa1ddfc6/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.092892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ebd5ea-6909-4068-b09a-d650ec3dc3b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5705979642f541aa9229c56505a6ee6e, group_id=2c680d1ff0f34e46a520cb30aa1ddfc6, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.095136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ebd5ea-6909-4068-b09a-d650ec3dc3b7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.115143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 223/423] 10.222.0.42 () {58 vars in 1442 bytes} [Thu Jul 21 22:19:40 2022] PUT /identity/v3/projects/5705979642f541aa9229c56505a6ee6e/groups/2c680d1ff0f34e46a520cb30aa1ddfc6/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:19:41.116075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebd5ea-6909-4068-b09a-d650ec3dc3b7 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:41.134097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec00d5-1769-47cd-9ea3-9878aa9749e2 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.177043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a20d1-8fd9-468a-9124-f40df722dd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.178051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a20d1-8fd9-468a-9124-f40df722dd53 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.178521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a20d1-8fd9-468a-9124-f40df722dd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.179059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a20d1-8fd9-468a-9124-f40df722dd53 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.180051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a20d1-8fd9-468a-9124-f40df722dd53 None admin] RBAC: Adding query filter params (name=member) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:41.180612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a20d1-8fd9-468a-9124-f40df722dd53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.182278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a20d1-8fd9-468a-9124-f40df722dd53 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.188495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:41.195968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:41.204526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 201/424] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:41.211078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a20d1-8fd9-468a-9124-f40df722dd53 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:41.320802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebd5ea-6909-4068-b09a-d650ec3dc3b7 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.381297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b629324-f14b-45bc-a625-6de2d85f051d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.384094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629324-f14b-45bc-a625-6de2d85f051d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.385624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629324-f14b-45bc-a625-6de2d85f051d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.386763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629324-f14b-45bc-a625-6de2d85f051d None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.391782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b629324-f14b-45bc-a625-6de2d85f051d None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.395195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b629324-f14b-45bc-a625-6de2d85f051d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.397714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 224/425] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:41.407453 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b629324-f14b-45bc-a625-6de2d85f051d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:41.493190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a20d1-8fd9-468a-9124-f40df722dd53 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.553139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b629324-f14b-45bc-a625-6de2d85f051d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.582054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:41.593984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c601cf2-4b7c-4bf3-a506-eb13260f7bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.595690 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c601cf2-4b7c-4bf3-a506-eb13260f7bcf None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.596611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c601cf2-4b7c-4bf3-a506-eb13260f7bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.597428 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c601cf2-4b7c-4bf3-a506-eb13260f7bcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.598281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6de1a-cb4b-4865-8a23-a9ff46c8fc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.603278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6de1a-cb4b-4865-8a23-a9ff46c8fc62 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.603278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6de1a-cb4b-4865-8a23-a9ff46c8fc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.603278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6de1a-cb4b-4865-8a23-a9ff46c8fc62 None admin] PATH_INFO: `/v3/groups/f2c406148932437fa57d1eed86c91879` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.603278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6de1a-cb4b-4865-8a23-a9ff46c8fc62 None admin] RBAC: Authorizing `identity:get_group(group_id=f2c406148932437fa57d1eed86c91879)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.603278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6de1a-cb4b-4865-8a23-a9ff46c8fc62 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.604829 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 225/426] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/groups/f2c406148932437fa57d1eed86c91879 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:41.613784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6de1a-cb4b-4865-8a23-a9ff46c8fc62 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:41.770035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6de1a-cb4b-4865-8a23-a9ff46c8fc62 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:41.807076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:41.811277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c601cf2-4b7c-4bf3-a506-eb13260f7bcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.816885 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c601cf2-4b7c-4bf3-a506-eb13260f7bcf None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.850998 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3d813-4113-44ed-9f6a-24d65fd4a1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:41.851665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3d813-4113-44ed-9f6a-24d65fd4a1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.851949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3d813-4113-44ed-9f6a-24d65fd4a1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.852229 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3d813-4113-44ed-9f6a-24d65fd4a1e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.853120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3d813-4113-44ed-9f6a-24d65fd4a1e7 None admin] RBAC: Adding query filter params (group.id=f2c406148932437fa57d1eed86c91879, role.id=80eea410deca46e2924f27afe5136d55, scope.project.id=bd637189b43c45c28839d066df463bc3) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:41.853442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3d813-4113-44ed-9f6a-24d65fd4a1e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:41.854909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3d813-4113-44ed-9f6a-24d65fd4a1e7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:41.862352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 226/427] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/role_assignments?group.id=f2c406148932437fa57d1eed86c91879&scope.project.id=bd637189b43c45c28839d066df463bc3&role.id=80eea410deca46e2924f27afe5136d55 => generated 611 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:41.863791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b474-1234-462f-bca9-a637bed52c87 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.864078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b474-1234-462f-bca9-a637bed52c87 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.864356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074b474-1234-462f-bca9-a637bed52c87 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.870190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 227/428] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:41.871991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e46ec-8f2f-410f-9c53-2a9a272b8633 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:41.872415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e46ec-8f2f-410f-9c53-2a9a272b8633 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:41.872851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e46ec-8f2f-410f-9c53-2a9a272b8633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:41.891383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d64e46ec-8f2f-410f-9c53-2a9a272b8633 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:41.892860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d64e46ec-8f2f-410f-9c53-2a9a272b8633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:41.944689 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 202/429] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 731 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:19:41.949710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c601cf2-4b7c-4bf3-a506-eb13260f7bcf None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:41.960568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:41.982076 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 228/430] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:42.005350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e46ec-8f2f-410f-9c53-2a9a272b8633 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:42.122533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e46ec-8f2f-410f-9c53-2a9a272b8633 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.163239 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a57f7-2839-4a97-a892-49bc3ac2015b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.164058 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a57f7-2839-4a97-a892-49bc3ac2015b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.164339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a57f7-2839-4a97-a892-49bc3ac2015b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.164675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a57f7-2839-4a97-a892-49bc3ac2015b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.177086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a57f7-2839-4a97-a892-49bc3ac2015b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:42.177498 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:42.184162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c601cf2-4b7c-4bf3-a506-eb13260f7bcf None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-d34a57f7-2839-4a97-a892-49bc3ac2015b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:42.191565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:42.194802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:42.197780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 229/431] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:42.200144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a57f7-2839-4a97-a892-49bc3ac2015b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:42.251659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:42.263636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9d0e6-9777-4268-9381-cf5af01bf292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.267720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9d0e6-9777-4268-9381-cf5af01bf292 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.268641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9d0e6-9777-4268-9381-cf5af01bf292 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.269537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9d0e6-9777-4268-9381-cf5af01bf292 None admin] PATH_INFO: `/v3/projects/bd637189b43c45c28839d066df463bc3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.272685 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9d0e6-9777-4268-9381-cf5af01bf292 None admin] RBAC: Authorizing `identity:get_project(project_id=bd637189b43c45c28839d066df463bc3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:42.277888 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9d0e6-9777-4268-9381-cf5af01bf292 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:42.283166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 203/432] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:41 2022] GET /identity/v3/projects/bd637189b43c45c28839d066df463bc3 => generated 298 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:42.293577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9d0e6-9777-4268-9381-cf5af01bf292 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:42.319186 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a57f7-2839-4a97-a892-49bc3ac2015b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.363879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbee0c-2322-4ada-abd5-36a37af77d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.364596 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbee0c-2322-4ada-abd5-36a37af77d2e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.364883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbee0c-2322-4ada-abd5-36a37af77d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.365159 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbee0c-2322-4ada-abd5-36a37af77d2e None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.365924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbee0c-2322-4ada-abd5-36a37af77d2e None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:42.366231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbee0c-2322-4ada-abd5-36a37af77d2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:42.367713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbee0c-2322-4ada-abd5-36a37af77d2e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:42.379509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 230/433] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:19:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:42.385061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbee0c-2322-4ada-abd5-36a37af77d2e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:42.563321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9d0e6-9777-4268-9381-cf5af01bf292 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.600079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbee0c-2322-4ada-abd5-36a37af77d2e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.616767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86c07f-4e4b-4754-abd2-8c075014d597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.617662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86c07f-4e4b-4754-abd2-8c075014d597 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.617945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86c07f-4e4b-4754-abd2-8c075014d597 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.618218 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86c07f-4e4b-4754-abd2-8c075014d597 None admin] PATH_INFO: `/v3/groups/f2c406148932437fa57d1eed86c91879` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.619512 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86c07f-4e4b-4754-abd2-8c075014d597 None admin] RBAC: Authorizing `identity:get_group(group_id=f2c406148932437fa57d1eed86c91879)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:42.621085 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86c07f-4e4b-4754-abd2-8c075014d597 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:42.621526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:42.629287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 204/434] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:42 2022] GET /identity/v3/groups/f2c406148932437fa57d1eed86c91879 => generated 226 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:42.630416 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86c07f-4e4b-4754-abd2-8c075014d597 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:42.679883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:42.706211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad730c-0e0c-46d2-92eb-35a900729cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.707675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad730c-0e0c-46d2-92eb-35a900729cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.708848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad730c-0e0c-46d2-92eb-35a900729cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.709183 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad730c-0e0c-46d2-92eb-35a900729cc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.798310 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86c07f-4e4b-4754-abd2-8c075014d597 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:42.816445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad730c-0e0c-46d2-92eb-35a900729cc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:42.817960 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad730c-0e0c-46d2-92eb-35a900729cc3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:42.872225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7259296-9925-4487-8d94-7c2187cb7d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:42.873271 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7259296-9925-4487-8d94-7c2187cb7d86 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:42.873888 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7259296-9925-4487-8d94-7c2187cb7d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:42.873888 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7259296-9925-4487-8d94-7c2187cb7d86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:42.875815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7259296-9925-4487-8d94-7c2187cb7d86 None admin] RBAC: Adding query filter params (group.id=f2c406148932437fa57d1eed86c91879, role.id=1b870be20b0c4159bfbbe73593799250, scope.project.id=bd637189b43c45c28839d066df463bc3) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:42.876145 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7259296-9925-4487-8d94-7c2187cb7d86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:42.878761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7259296-9925-4487-8d94-7c2187cb7d86 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:42.895879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 205/435] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:42 2022] GET /identity/v3/role_assignments?group.id=f2c406148932437fa57d1eed86c91879&scope.project.id=bd637189b43c45c28839d066df463bc3&role.id=1b870be20b0c4159bfbbe73593799250 => generated 611 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:43.038001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 231/436] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:19:43.047432 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7259296-9925-4487-8d94-7c2187cb7d86 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:43.249351 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:43.255456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7259296-9925-4487-8d94-7c2187cb7d86 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:43.332103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccea2c-314d-46f2-ac55-dfa37144c409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:43.333672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccea2c-314d-46f2-ac55-dfa37144c409 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.334145 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccea2c-314d-46f2-ac55-dfa37144c409 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.334628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccea2c-314d-46f2-ac55-dfa37144c409 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.337017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccea2c-314d-46f2-ac55-dfa37144c409 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:43.339798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccea2c-314d-46f2-ac55-dfa37144c409 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:43.345456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 206/437] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:43 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:43.351622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad730c-0e0c-46d2-92eb-35a900729cc3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:43.364407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:43.396136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28d30a-4551-4f0f-8b70-711521ca2a05 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.396851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28d30a-4551-4f0f-8b70-711521ca2a05 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.398923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28d30a-4551-4f0f-8b70-711521ca2a05 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.401348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 207/438] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:19:43.412161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226203d-121f-4cf8-b1a2-0a62ecc7ed81 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.413053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226203d-121f-4cf8-b1a2-0a62ecc7ed81 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.413772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226203d-121f-4cf8-b1a2-0a62ecc7ed81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.444528 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-1226203d-121f-4cf8-b1a2-0a62ecc7ed81 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:43.446304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1226203d-121f-4cf8-b1a2-0a62ecc7ed81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:43.560178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad730c-0e0c-46d2-92eb-35a900729cc3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:43.573549 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 208/439] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:43.617269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226203d-121f-4cf8-b1a2-0a62ecc7ed81 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:43.636328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244d1c3-8030-4893-a60f-540f19ab6311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:43.637888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244d1c3-8030-4893-a60f-540f19ab6311 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.638396 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244d1c3-8030-4893-a60f-540f19ab6311 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.639781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244d1c3-8030-4893-a60f-540f19ab6311 None admin] PATH_INFO: `/v3/groups/f2c406148932437fa57d1eed86c91879` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.641679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244d1c3-8030-4893-a60f-540f19ab6311 None admin] RBAC: Authorizing `identity:get_group(group_id=f2c406148932437fa57d1eed86c91879)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:43.645999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244d1c3-8030-4893-a60f-540f19ab6311 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:43.648516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 232/440] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:43 2022] GET /identity/v3/groups/f2c406148932437fa57d1eed86c91879 => generated 226 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:43.659911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244d1c3-8030-4893-a60f-540f19ab6311 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:43.791932 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:43.799447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226203d-121f-4cf8-b1a2-0a62ecc7ed81 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:43.809036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:43.855811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244d1c3-8030-4893-a60f-540f19ab6311 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:43.858383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63f875-955c-4e2e-93b8-78e1f94c27f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:43.859534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63f875-955c-4e2e-93b8-78e1f94c27f0 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.859941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63f875-955c-4e2e-93b8-78e1f94c27f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.860342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63f875-955c-4e2e-93b8-78e1f94c27f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.874282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63f875-955c-4e2e-93b8-78e1f94c27f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-0a63f875-955c-4e2e-93b8-78e1f94c27f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:43.882804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:43.884471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:43.885833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:43.885833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:43.885833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:43.885833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:43.885833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:43.885833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:43.885833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:43.885833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:43.885833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:43.885833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:43.885833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:43.885833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:43.888675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 209/441] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:43.890730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3cff5-9d78-431e-9a87-409d90aca40d None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.891023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3cff5-9d78-431e-9a87-409d90aca40d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.891308 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3cff5-9d78-431e-9a87-409d90aca40d None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.892757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 210/442] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:43.893511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3cff5-9d78-431e-9a87-409d90aca40d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:43.921281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251a85d-b4f3-44d5-ad5f-7bee45811fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:43.922238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251a85d-b4f3-44d5-ad5f-7bee45811fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.922690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251a85d-b4f3-44d5-ad5f-7bee45811fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.923131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251a85d-b4f3-44d5-ad5f-7bee45811fb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.924647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251a85d-b4f3-44d5-ad5f-7bee45811fb6 None admin] RBAC: Adding query filter params (group.id=f2c406148932437fa57d1eed86c91879, role.id=1b870be20b0c4159bfbbe73593799250, scope.project.id=8f4b4898eca3450daecc84800ff218c9) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:43.925158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251a85d-b4f3-44d5-ad5f-7bee45811fb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:43.927359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251a85d-b4f3-44d5-ad5f-7bee45811fb6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:43.936642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 233/443] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:43 2022] GET /identity/v3/role_assignments?group.id=f2c406148932437fa57d1eed86c91879&scope.project.id=8f4b4898eca3450daecc84800ff218c9&role.id=1b870be20b0c4159bfbbe73593799250 => generated 611 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:43.938053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9ac1b-0d1f-4f04-b9d9-00a3dbdcab48 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:43.938383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9ac1b-0d1f-4f04-b9d9-00a3dbdcab48 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:43.938799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9ac1b-0d1f-4f04-b9d9-00a3dbdcab48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:43.968810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1fc9ac1b-0d1f-4f04-b9d9-00a3dbdcab48 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:43.970080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1fc9ac1b-0d1f-4f04-b9d9-00a3dbdcab48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:44.048847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3cff5-9d78-431e-9a87-409d90aca40d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:44.055469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 234/444] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:44.085939 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9ac1b-0d1f-4f04-b9d9-00a3dbdcab48 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:44.097328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04b7e6-488f-4f75-b4a2-788ccf5946c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:44.098137 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04b7e6-488f-4f75-b4a2-788ccf5946c2 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.098458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04b7e6-488f-4f75-b4a2-788ccf5946c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.098776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04b7e6-488f-4f75-b4a2-788ccf5946c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.099780 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04b7e6-488f-4f75-b4a2-788ccf5946c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:44.100280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04b7e6-488f-4f75-b4a2-788ccf5946c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:44.102043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04b7e6-488f-4f75-b4a2-788ccf5946c2 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:44.119080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 211/445] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:44.120594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04b7e6-488f-4f75-b4a2-788ccf5946c2 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:44.150481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:44.193900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:44.244026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9ac1b-0d1f-4f04-b9d9-00a3dbdcab48 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:44.283516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04b7e6-488f-4f75-b4a2-788ccf5946c2 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:44.309238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b116d79-995f-415f-827a-1ddf9984ea31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:44.310125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b116d79-995f-415f-827a-1ddf9984ea31 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.310411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b116d79-995f-415f-827a-1ddf9984ea31 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.310726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b116d79-995f-415f-827a-1ddf9984ea31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.324522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b116d79-995f-415f-827a-1ddf9984ea31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:44.332003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08919493-f966-4466-b995-1c35a5e0df94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:44.332803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08919493-f966-4466-b995-1c35a5e0df94 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.333089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08919493-f966-4466-b995-1c35a5e0df94 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-4b116d79-995f-415f-827a-1ddf9984ea31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:44.333190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:44.334047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:44.334838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:44.334838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:44.334838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:44.334838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:44.334838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:44.334838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:44.334838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:44.334838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:44.334838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:44.334838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:44.334838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:19:44.334838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:44.335471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08919493-f966-4466-b995-1c35a5e0df94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.335547 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 235/446] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:19:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:44.344168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b116d79-995f-415f-827a-1ddf9984ea31 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:44.438502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08919493-f966-4466-b995-1c35a5e0df94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:44.439964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08919493-f966-4466-b995-1c35a5e0df94 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:44.463050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:44.492029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b116d79-995f-415f-827a-1ddf9984ea31 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:44.519938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:44.530887 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af894111-4612-4951-94ae-5ff5355931e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:44.531729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af894111-4612-4951-94ae-5ff5355931e2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.532023 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af894111-4612-4951-94ae-5ff5355931e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.532305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af894111-4612-4951-94ae-5ff5355931e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.533125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af894111-4612-4951-94ae-5ff5355931e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:44.533439 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af894111-4612-4951-94ae-5ff5355931e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:44.534850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af894111-4612-4951-94ae-5ff5355931e2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:44.544183 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 236/447] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:44.550699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af894111-4612-4951-94ae-5ff5355931e2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:44.562204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 212/448] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:44.568843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08919493-f966-4466-b995-1c35a5e0df94 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:44.662124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af894111-4612-4951-94ae-5ff5355931e2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:44.681566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08919493-f966-4466-b995-1c35a5e0df94 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:44.700171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900298a-020d-46c6-a41b-53507015ec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:44.700775 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900298a-020d-46c6-a41b-53507015ec54 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.701062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900298a-020d-46c6-a41b-53507015ec54 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.701366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900298a-020d-46c6-a41b-53507015ec54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.706964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:44.726115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a95243-86b9-4071-aef1-1a768919d348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:44.727509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a95243-86b9-4071-aef1-1a768919d348 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.727969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a95243-86b9-4071-aef1-1a768919d348 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.728418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a95243-86b9-4071-aef1-1a768919d348 None admin] PATH_INFO: `/v3/projects/8f4b4898eca3450daecc84800ff218c9` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.730252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a95243-86b9-4071-aef1-1a768919d348 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4b4898eca3450daecc84800ff218c9)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:44.732986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a95243-86b9-4071-aef1-1a768919d348 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:44.736034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 213/449] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:44 2022] GET /identity/v3/projects/8f4b4898eca3450daecc84800ff218c9 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:44.741756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a95243-86b9-4071-aef1-1a768919d348 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:44.776154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:44.789548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900298a-020d-46c6-a41b-53507015ec54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:44.791018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900298a-020d-46c6-a41b-53507015ec54 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:44.854491 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a95243-86b9-4071-aef1-1a768919d348 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:44.893855 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60839f27-4ce4-4649-bfa2-d8abd3814b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:44.894760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60839f27-4ce4-4649-bfa2-d8abd3814b37 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:44.895051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60839f27-4ce4-4649-bfa2-d8abd3814b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:44.895330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60839f27-4ce4-4649-bfa2-d8abd3814b37 None admin] PATH_INFO: `/v3/groups/f2c406148932437fa57d1eed86c91879` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:44.896980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60839f27-4ce4-4649-bfa2-d8abd3814b37 None admin] RBAC: Authorizing `identity:get_group(group_id=f2c406148932437fa57d1eed86c91879)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:44.898490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60839f27-4ce4-4649-bfa2-d8abd3814b37 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:44.900248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 214/450] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:44 2022] GET /identity/v3/groups/f2c406148932437fa57d1eed86c91879 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:44.905746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 237/451] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:19:44.906667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60839f27-4ce4-4649-bfa2-d8abd3814b37 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:44.912362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900298a-020d-46c6-a41b-53507015ec54 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:44.980222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:45.035968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:45.040123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60839f27-4ce4-4649-bfa2-d8abd3814b37 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:45.048365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900298a-020d-46c6-a41b-53507015ec54 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:45.081690 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c47484-c742-4830-a670-238af20f62e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:45.082422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c47484-c742-4830-a670-238af20f62e6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:45.082662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c47484-c742-4830-a670-238af20f62e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:45.082935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c47484-c742-4830-a670-238af20f62e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:45.083797 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c47484-c742-4830-a670-238af20f62e6 None admin] RBAC: Adding query filter params (group.id=f2c406148932437fa57d1eed86c91879, role.id=80eea410deca46e2924f27afe5136d55, scope.project.id=8f4b4898eca3450daecc84800ff218c9) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:45.084113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c47484-c742-4830-a670-238af20f62e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:45.085611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c47484-c742-4830-a670-238af20f62e6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:45.088278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e09412-d593-482f-8539-f1d26c2aa1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:45.089169 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e09412-d593-482f-8539-f1d26c2aa1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:45.089446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e09412-d593-482f-8539-f1d26c2aa1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:45.089715 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e09412-d593-482f-8539-f1d26c2aa1e0 None admin] PATH_INFO: `/v3/projects/5705979642f541aa9229c56505a6ee6e` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:45.090931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e09412-d593-482f-8539-f1d26c2aa1e0 None admin] RBAC: Authorizing `identity:get_project(project_id=5705979642f541aa9229c56505a6ee6e)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:45.092738 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e09412-d593-482f-8539-f1d26c2aa1e0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:45.092914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 215/452] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:44 2022] GET /identity/v3/role_assignments?group.id=f2c406148932437fa57d1eed86c91879&scope.project.id=8f4b4898eca3450daecc84800ff218c9&role.id=80eea410deca46e2924f27afe5136d55 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:45.094326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 238/453] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:19:44 2022] GET /identity/v3/projects/5705979642f541aa9229c56505a6ee6e => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:19:45.100141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c47484-c742-4830-a670-238af20f62e6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:45.222123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c47484-c742-4830-a670-238af20f62e6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:45.248599 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:45.265688 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424be243-c3a5-44a6-a0e3-8ba1356b7dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:45.266583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424be243-c3a5-44a6-a0e3-8ba1356b7dce None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:45.266868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424be243-c3a5-44a6-a0e3-8ba1356b7dce None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:45.267149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424be243-c3a5-44a6-a0e3-8ba1356b7dce None admin] PATH_INFO: `/v3/groups/2c680d1ff0f34e46a520cb30aa1ddfc6` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:45.268428 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424be243-c3a5-44a6-a0e3-8ba1356b7dce None admin] RBAC: Authorizing `identity:get_group(group_id=2c680d1ff0f34e46a520cb30aa1ddfc6)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:45.269941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424be243-c3a5-44a6-a0e3-8ba1356b7dce None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:45.271990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 216/454] 10.222.0.42 () {56 vars in 1180 bytes} [Thu Jul 21 22:19:45 2022] GET /identity/v3/groups/2c680d1ff0f34e46a520cb30aa1ddfc6 => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:45.278220 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e09412-d593-482f-8539-f1d26c2aa1e0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:45.400631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e09412-d593-482f-8539-f1d26c2aa1e0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:45.440412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c159f-eec7-491b-b643-3f9aad32da59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:45.441182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c159f-eec7-491b-b643-3f9aad32da59 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:45.441468 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c159f-eec7-491b-b643-3f9aad32da59 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:45.441740 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c159f-eec7-491b-b643-3f9aad32da59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:45.442577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c159f-eec7-491b-b643-3f9aad32da59 None admin] RBAC: Adding query filter params (group.id=2c680d1ff0f34e46a520cb30aa1ddfc6, role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=5705979642f541aa9229c56505a6ee6e) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:45.442886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c159f-eec7-491b-b643-3f9aad32da59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:45.444346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c159f-eec7-491b-b643-3f9aad32da59 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:45.452526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 239/455] 10.222.0.42 () {56 vars in 1376 bytes} [Thu Jul 21 22:19:45 2022] GET /identity/v3/role_assignments?group.id=2c680d1ff0f34e46a520cb30aa1ddfc6&scope.project.id=5705979642f541aa9229c56505a6ee6e&role.id=f957e09d047b47cfa0891d25b4181263 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:48.154940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c431822-475e-4e81-a43d-47a93027196d None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:48.156164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c431822-475e-4e81-a43d-47a93027196d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:48.156746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c431822-475e-4e81-a43d-47a93027196d None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:48.158718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 217/456] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:19:48.164735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149fc30-141e-48b2-a47b-b87ee09c62cd None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:48.165261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149fc30-141e-48b2-a47b-b87ee09c62cd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:48.165261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149fc30-141e-48b2-a47b-b87ee09c62cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:48.185980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9149fc30-141e-48b2-a47b-b87ee09c62cd None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:48.187389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9149fc30-141e-48b2-a47b-b87ee09c62cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:48.217841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:48.316781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 240/457] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:48.353452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c431822-475e-4e81-a43d-47a93027196d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:48.555158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c431822-475e-4e81-a43d-47a93027196d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:48.641670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c637bf-1241-4446-aac4-1f270b07de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:48.642993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c637bf-1241-4446-aac4-1f270b07de27 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:48.643474 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c637bf-1241-4446-aac4-1f270b07de27 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:48.643957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c637bf-1241-4446-aac4-1f270b07de27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:48.672001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c637bf-1241-4446-aac4-1f270b07de27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:48.674772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c637bf-1241-4446-aac4-1f270b07de27 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-c3c637bf-1241-4446-aac4-1f270b07de27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:48.722535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:48.724214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:48.726850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:48.726850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:48.726850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:48.726850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:48.726850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 218/458] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:48.734071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149fc30-141e-48b2-a47b-b87ee09c62cd None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:48.916435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149fc30-141e-48b2-a47b-b87ee09c62cd None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:48.975362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ffc4d-19d0-4982-b63f-29aed6e31849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:48.975948 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ffc4d-19d0-4982-b63f-29aed6e31849 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:48.976231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ffc4d-19d0-4982-b63f-29aed6e31849 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:48.976531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ffc4d-19d0-4982-b63f-29aed6e31849 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:48.977289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ffc4d-19d0-4982-b63f-29aed6e31849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:48.977594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ffc4d-19d0-4982-b63f-29aed6e31849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:48.979010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ffc4d-19d0-4982-b63f-29aed6e31849 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:48.996438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 241/459] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:49.002913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c637bf-1241-4446-aac4-1f270b07de27 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:49.132911 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:49.285632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c637bf-1241-4446-aac4-1f270b07de27 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:49.371663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c580e4d-d420-4402-8905-5ec6bcb95b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:49.372362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c580e4d-d420-4402-8905-5ec6bcb95b76 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.372697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c580e4d-d420-4402-8905-5ec6bcb95b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.373848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c580e4d-d420-4402-8905-5ec6bcb95b76 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.374707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c580e4d-d420-4402-8905-5ec6bcb95b76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:49.376202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c580e4d-d420-4402-8905-5ec6bcb95b76 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:49.417237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 219/460] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:49 2022] POST /identity/v3/users => generated 246 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:49.460648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0ba1b-748b-4e40-8709-31012bc003e3 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.461442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0ba1b-748b-4e40-8709-31012bc003e3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.461442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0ba1b-748b-4e40-8709-31012bc003e3 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.463008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 242/461] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:49.471036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee1500-6ec9-4544-92c4-9a9219d4fe97 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.471368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee1500-6ec9-4544-92c4-9a9219d4fe97 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.471591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee1500-6ec9-4544-92c4-9a9219d4fe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.486759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eac47-6152-4f7b-9bd4-b740ccffaefb None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.487473 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eac47-6152-4f7b-9bd4-b740ccffaefb None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.487991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eac47-6152-4f7b-9bd4-b740ccffaefb None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.489767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 243/462] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:19:49.502909 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-c4ee1500-6ec9-4544-92c4-9a9219d4fe97 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:49.504845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c4ee1500-6ec9-4544-92c4-9a9219d4fe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:49.507874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552d056-fcdb-435f-93e1-761eda033641 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.508814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552d056-fcdb-435f-93e1-761eda033641 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.509566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552d056-fcdb-435f-93e1-761eda033641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.554249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d552d056-fcdb-435f-93e1-761eda033641 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:49.559108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d552d056-fcdb-435f-93e1-761eda033641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:49.589247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:49.650067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 220/463] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:19:49.651527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d873c2-48d9-43a6-9d1b-bc694ab10fd8 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.651813 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d873c2-48d9-43a6-9d1b-bc694ab10fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.652323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d873c2-48d9-43a6-9d1b-bc694ab10fd8 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.659218 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 221/464] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:49.661052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97176733-f5ad-4b3d-b1fb-234ab92a629f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.661606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97176733-f5ad-4b3d-b1fb-234ab92a629f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.662046 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97176733-f5ad-4b3d-b1fb-234ab92a629f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.681097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-97176733-f5ad-4b3d-b1fb-234ab92a629f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:49.682716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-97176733-f5ad-4b3d-b1fb-234ab92a629f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:49.725822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:49.728582 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 244/465] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:49.729543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552d056-fcdb-435f-93e1-761eda033641 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:49.769937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 222/466] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:49.777760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97176733-f5ad-4b3d-b1fb-234ab92a629f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:49.865429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552d056-fcdb-435f-93e1-761eda033641 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:49.892272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97176733-f5ad-4b3d-b1fb-234ab92a629f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:49.907294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e901c48-5ff3-4402-aaea-e1cc807a203f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:49.908322 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e901c48-5ff3-4402-aaea-e1cc807a203f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.908735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e901c48-5ff3-4402-aaea-e1cc807a203f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.909105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e901c48-5ff3-4402-aaea-e1cc807a203f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.921143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e901c48-5ff3-4402-aaea-e1cc807a203f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:49.922835 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e901c48-5ff3-4402-aaea-e1cc807a203f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:49.925074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:49.934320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502d89a-4c21-4613-8559-2cdb1ab7c16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:49.936102 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502d89a-4c21-4613-8559-2cdb1ab7c16f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:49.936697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502d89a-4c21-4613-8559-2cdb1ab7c16f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:49.937130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502d89a-4c21-4613-8559-2cdb1ab7c16f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-5e901c48-5ff3-4402-aaea-e1cc807a203f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:49.946686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:49.947660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:49.948552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:49.948552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:49.948552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:49.948552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:49.948552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 245/467] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:49.951434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502d89a-4c21-4613-8559-2cdb1ab7c16f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:49.953214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502d89a-4c21-4613-8559-2cdb1ab7c16f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:49.953586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e901c48-5ff3-4402-aaea-e1cc807a203f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-1502d89a-4c21-4613-8559-2cdb1ab7c16f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:49.974634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:49.977344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:49.979028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:49.979028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:49.979028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:49.979028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:49.979028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 223/468] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:49.979028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502d89a-4c21-4613-8559-2cdb1ab7c16f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:49.989204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:50.121925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e901c48-5ff3-4402-aaea-e1cc807a203f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:50.123568 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502d89a-4c21-4613-8559-2cdb1ab7c16f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:50.171702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39daac-446a-4ec8-bfa2-5d6474c27026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:50.172514 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39daac-446a-4ec8-bfa2-5d6474c27026 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.172801 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39daac-446a-4ec8-bfa2-5d6474c27026 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.173079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39daac-446a-4ec8-bfa2-5d6474c27026 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.173822 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832344dc-523a-4e58-a1fc-1505b4cd2194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:50.174011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39daac-446a-4ec8-bfa2-5d6474c27026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:50.174153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39daac-446a-4ec8-bfa2-5d6474c27026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:50.174866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832344dc-523a-4e58-a1fc-1505b4cd2194 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.175278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832344dc-523a-4e58-a1fc-1505b4cd2194 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.175526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39daac-446a-4ec8-bfa2-5d6474c27026 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:50.175769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832344dc-523a-4e58-a1fc-1505b4cd2194 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.189514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832344dc-523a-4e58-a1fc-1505b4cd2194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:50.195718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 224/469] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:50.195909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832344dc-523a-4e58-a1fc-1505b4cd2194 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:50.196002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7949575-e33f-4052-9e72-d4be1ded94ae None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.196002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7949575-e33f-4052-9e72-d4be1ded94ae None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.196002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7949575-e33f-4052-9e72-d4be1ded94ae None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.196515 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 225/470] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:50.197819 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7949575-e33f-4052-9e72-d4be1ded94ae None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-832344dc-523a-4e58-a1fc-1505b4cd2194 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:50.220568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:50.222139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:50.223785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:50.223785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:50.223785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:50.223785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:50.223785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 246/471] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:50.224965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832344dc-523a-4e58-a1fc-1505b4cd2194 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:50.287914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:50.308294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:50.372273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7949575-e33f-4052-9e72-d4be1ded94ae None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:50.391008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832344dc-523a-4e58-a1fc-1505b4cd2194 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:50.437629 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00603fc-a50e-4736-b084-0545f06fbdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:50.438324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00603fc-a50e-4736-b084-0545f06fbdad None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.438612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00603fc-a50e-4736-b084-0545f06fbdad None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.438892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00603fc-a50e-4736-b084-0545f06fbdad None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.439767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00603fc-a50e-4736-b084-0545f06fbdad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:50.439962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00603fc-a50e-4736-b084-0545f06fbdad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:50.441429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00603fc-a50e-4736-b084-0545f06fbdad None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:50.447266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69826735-25c5-4b7f-b2ed-21bad5aaee4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:50.448123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69826735-25c5-4b7f-b2ed-21bad5aaee4f None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.448568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69826735-25c5-4b7f-b2ed-21bad5aaee4f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.448996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69826735-25c5-4b7f-b2ed-21bad5aaee4f None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.450246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69826735-25c5-4b7f-b2ed-21bad5aaee4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:50.451901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69826735-25c5-4b7f-b2ed-21bad5aaee4f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:50.459530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 226/472] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:50.460760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937aead-f652-4638-87d3-190e1ee1b61d None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.461051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937aead-f652-4638-87d3-190e1ee1b61d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.461335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937aead-f652-4638-87d3-190e1ee1b61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.480688 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 247/473] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:50 2022] POST /identity/v3/users => generated 248 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:50.484069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69826735-25c5-4b7f-b2ed-21bad5aaee4f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:50.490736 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-a937aead-f652-4638-87d3-190e1ee1b61d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:50.492003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a937aead-f652-4638-87d3-190e1ee1b61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:50.648186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 227/474] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:50.649800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937aead-f652-4638-87d3-190e1ee1b61d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:50.687082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69826735-25c5-4b7f-b2ed-21bad5aaee4f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:50.690193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:50.768332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf49057-22f8-4370-a938-5d44cf0a6188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:50.769980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf49057-22f8-4370-a938-5d44cf0a6188 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.771922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf49057-22f8-4370-a938-5d44cf0a6188 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.772390 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf49057-22f8-4370-a938-5d44cf0a6188 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.773789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf49057-22f8-4370-a938-5d44cf0a6188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:50.774322 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf49057-22f8-4370-a938-5d44cf0a6188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:50.776938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf49057-22f8-4370-a938-5d44cf0a6188 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:50.797784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:50.801067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 248/475] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:50.801936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf49057-22f8-4370-a938-5d44cf0a6188 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:50.873151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937aead-f652-4638-87d3-190e1ee1b61d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:50.928525 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b82980-548d-4178-81cd-93c7bfe2098b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:50.929313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b82980-548d-4178-81cd-93c7bfe2098b None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:50.929602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b82980-548d-4178-81cd-93c7bfe2098b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:50.929881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b82980-548d-4178-81cd-93c7bfe2098b None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:50.931046 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b82980-548d-4178-81cd-93c7bfe2098b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:50.932814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b82980-548d-4178-81cd-93c7bfe2098b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:50.960611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 228/476] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:50 2022] POST /identity/v3/users => generated 248 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:50.961860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b82980-548d-4178-81cd-93c7bfe2098b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:51.009594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf49057-22f8-4370-a938-5d44cf0a6188 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:51.084223 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48b600-948a-418b-80ba-915aff15289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:51.085857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48b600-948a-418b-80ba-915aff15289f None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:51.086387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48b600-948a-418b-80ba-915aff15289f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:51.086892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48b600-948a-418b-80ba-915aff15289f None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:51.088460 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48b600-948a-418b-80ba-915aff15289f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:51.092710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48b600-948a-418b-80ba-915aff15289f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:51.105896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.sql.core [None req-2c48b600-948a-418b-80ba-915aff15289f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 22:19:51.105896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 22:19:51.105896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [parameters: {'id': '1095078711b649afb4ae230e0e38c2d2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 22:19:51.105896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80315) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 22:19:51.118411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b82980-548d-4178-81cd-93c7bfe2098b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:51.139147 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-2c48b600-948a-418b-80ba-915aff15289f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result.read() Jul 21 22:19:51.148099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:19:51.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application yield resource Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise self.value Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application session.commit() Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 22:19:51.150488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.session.flush() Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application compat.raise_( Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 22:19:51.151881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:19:51.153261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result.read() Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:19:51.155092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [parameters: {'id': '1095078711b649afb4ae230e0e38c2d2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:51.156070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 22:19:51.162900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:51.164158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 249/477] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jul 21 22:19:50 2022] POST /identity/v3/roles => generated 144 bytes in 349 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:51.164158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48b600-948a-418b-80ba-915aff15289f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:51.182872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4bb408-d3fa-4e68-bf51-0d2b0b317cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:51.183513 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bb408-d3fa-4e68-bf51-0d2b0b317cc0 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:51.183798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bb408-d3fa-4e68-bf51-0d2b0b317cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:51.184077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bb408-d3fa-4e68-bf51-0d2b0b317cc0 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:51.185053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4bb408-d3fa-4e68-bf51-0d2b0b317cc0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:51.187490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4bb408-d3fa-4e68-bf51-0d2b0b317cc0 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:51.234735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:51.239343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 229/478] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:50 2022] POST /identity/v3/users => generated 249 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:51.398268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48b600-948a-418b-80ba-915aff15289f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:51.471536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b5ea2-4b0b-42d7-948c-1f614fe49889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:51.472879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b5ea2-4b0b-42d7-948c-1f614fe49889 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:51.473366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b5ea2-4b0b-42d7-948c-1f614fe49889 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:51.473851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b5ea2-4b0b-42d7-948c-1f614fe49889 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:51.495721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b5ea2-4b0b-42d7-948c-1f614fe49889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-743b5ea2-4b0b-42d7-948c-1f614fe49889 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:51.512804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:51.515106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:51.518019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 250/479] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:19:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:51.522987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4bb408-d3fa-4e68-bf51-0d2b0b317cc0 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:51.776803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4bb408-d3fa-4e68-bf51-0d2b0b317cc0 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:51.869743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569177a9-913e-4447-8377-b4d7d3e5bdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:51.871394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569177a9-913e-4447-8377-b4d7d3e5bdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:51.872173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569177a9-913e-4447-8377-b4d7d3e5bdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:51.873075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569177a9-913e-4447-8377-b4d7d3e5bdf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:51.874878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569177a9-913e-4447-8377-b4d7d3e5bdf6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:51.875771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569177a9-913e-4447-8377-b4d7d3e5bdf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:51.878879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569177a9-913e-4447-8377-b4d7d3e5bdf6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:51.902441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 230/480] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:52.225180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94db781-cf17-45dd-86e8-e91c88ab91e5 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:52.225704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94db781-cf17-45dd-86e8-e91c88ab91e5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:52.226106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94db781-cf17-45dd-86e8-e91c88ab91e5 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:52.229044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 251/481] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:52.237553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a0da1-bc67-47c9-8b4b-0b27b98df389 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:52.237958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a0da1-bc67-47c9-8b4b-0b27b98df389 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:52.238217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a0da1-bc67-47c9-8b4b-0b27b98df389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:52.258383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-3f0a0da1-bc67-47c9-8b4b-0b27b98df389 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:52.259614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3f0a0da1-bc67-47c9-8b4b-0b27b98df389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:52.293516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:52.353597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 231/482] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:52.410889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94db781-cf17-45dd-86e8-e91c88ab91e5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:52.649375 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:52.692734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94db781-cf17-45dd-86e8-e91c88ab91e5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:52.767891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25298b8a-fc44-4f80-b7d0-22b1bb73711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:52.769784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25298b8a-fc44-4f80-b7d0-22b1bb73711f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:52.770532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25298b8a-fc44-4f80-b7d0-22b1bb73711f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:52.771568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25298b8a-fc44-4f80-b7d0-22b1bb73711f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:52.804019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25298b8a-fc44-4f80-b7d0-22b1bb73711f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-25298b8a-fc44-4f80-b7d0-22b1bb73711f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:52.825848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:52.829234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:52.830702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:52.830702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:52.830702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:52.830702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:52.830702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:52.830702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:52.830702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:52.830702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:52.830702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:52.830702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:52.830702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:52.830702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:52.835664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 252/483] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:52.843527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a0da1-bc67-47c9-8b4b-0b27b98df389 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:53.072104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a0da1-bc67-47c9-8b4b-0b27b98df389 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:53.169345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee054f-7e88-4cfe-9ed6-fcf2dd4cce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:53.170810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee054f-7e88-4cfe-9ed6-fcf2dd4cce1c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:53.171080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee054f-7e88-4cfe-9ed6-fcf2dd4cce1c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:53.171563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee054f-7e88-4cfe-9ed6-fcf2dd4cce1c None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:53.173084 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee054f-7e88-4cfe-9ed6-fcf2dd4cce1c None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:53.173620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee054f-7e88-4cfe-9ed6-fcf2dd4cce1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:53.176607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee054f-7e88-4cfe-9ed6-fcf2dd4cce1c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:53.204424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 232/484] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:53.210447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25298b8a-fc44-4f80-b7d0-22b1bb73711f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:53.468262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25298b8a-fc44-4f80-b7d0-22b1bb73711f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:53.550152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12defe7e-5992-4252-9d03-4ee2afcd6b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:53.552160 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12defe7e-5992-4252-9d03-4ee2afcd6b95 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:53.553081 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12defe7e-5992-4252-9d03-4ee2afcd6b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:53.553849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12defe7e-5992-4252-9d03-4ee2afcd6b95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:53.576598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12defe7e-5992-4252-9d03-4ee2afcd6b95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:53.579965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12defe7e-5992-4252-9d03-4ee2afcd6b95 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:53.593658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-12defe7e-5992-4252-9d03-4ee2afcd6b95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:53.618783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:53.620442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:53.622148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:53.622148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:53.622148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:53.622148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:53.627013 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 253/485] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:53.639148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee054f-7e88-4cfe-9ed6-fcf2dd4cce1c None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:53.719379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:53.897193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee054f-7e88-4cfe-9ed6-fcf2dd4cce1c None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:53.980418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77b267-7463-4ba9-9922-03c08793f36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:53.981453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77b267-7463-4ba9-9922-03c08793f36c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:53.981928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77b267-7463-4ba9-9922-03c08793f36c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:53.982372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77b267-7463-4ba9-9922-03c08793f36c None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:53.983673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77b267-7463-4ba9-9922-03c08793f36c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:53.984163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77b267-7463-4ba9-9922-03c08793f36c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:53.986563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77b267-7463-4ba9-9922-03c08793f36c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:54.008078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 233/486] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:54.017086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12defe7e-5992-4252-9d03-4ee2afcd6b95 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:54.230878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12defe7e-5992-4252-9d03-4ee2afcd6b95 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:54.283645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf78994-ef50-41da-bcc8-e27503381429 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.284152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf78994-ef50-41da-bcc8-e27503381429 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.284696 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf78994-ef50-41da-bcc8-e27503381429 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.287500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 234/487] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:54.293491 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2bd3-9a8b-4c64-bdc9-ea02f3bf54ca None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.294106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2bd3-9a8b-4c64-bdc9-ea02f3bf54ca None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.295140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2bd3-9a8b-4c64-bdc9-ea02f3bf54ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.313011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3c68a-738f-4ed1-aa3c-ee3e7ad61f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:54.314382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3c68a-738f-4ed1-aa3c-ee3e7ad61f55 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.314890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3c68a-738f-4ed1-aa3c-ee3e7ad61f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.315389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3c68a-738f-4ed1-aa3c-ee3e7ad61f55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.345703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f31e2bd3-9a8b-4c64-bdc9-ea02f3bf54ca None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:54.347053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f31e2bd3-9a8b-4c64-bdc9-ea02f3bf54ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:54.349769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3c68a-738f-4ed1-aa3c-ee3e7ad61f55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:54.352374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3c68a-738f-4ed1-aa3c-ee3e7ad61f55 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-eae3c68a-738f-4ed1-aa3c-ee3e7ad61f55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:54.408380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:54.409412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:54.410318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:54.410318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:54.410318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:54.410318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:54.410318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:54.410318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:54.410318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:54.410318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:54.410318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:19:54.410318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:54.412516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 254/488] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 22:19:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:54.422140 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3c68a-738f-4ed1-aa3c-ee3e7ad61f55 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:54.551867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 235/489] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:54.585708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:54.596677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e2bd3-9a8b-4c64-bdc9-ea02f3bf54ca None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:54.625872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:54.670911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3c68a-738f-4ed1-aa3c-ee3e7ad61f55 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:54.740625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013dcea-5431-49a2-97d7-99f14621a2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:54.741697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013dcea-5431-49a2-97d7-99f14621a2ce None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.742182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013dcea-5431-49a2-97d7-99f14621a2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.742844 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013dcea-5431-49a2-97d7-99f14621a2ce None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.743914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013dcea-5431-49a2-97d7-99f14621a2ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:54.746579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013dcea-5431-49a2-97d7-99f14621a2ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:54.748417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013dcea-5431-49a2-97d7-99f14621a2ce None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:54.796585 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 255/490] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 22:19:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:54.805373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013dcea-5431-49a2-97d7-99f14621a2ce None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:54.810240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e2bd3-9a8b-4c64-bdc9-ea02f3bf54ca None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:54.902985 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebe2b8-020e-4395-8dfc-d6fe717d8cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:54.905334 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebe2b8-020e-4395-8dfc-d6fe717d8cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.905826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebe2b8-020e-4395-8dfc-d6fe717d8cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.906303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebe2b8-020e-4395-8dfc-d6fe717d8cd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.924465 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebe2b8-020e-4395-8dfc-d6fe717d8cd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-3aebe2b8-020e-4395-8dfc-d6fe717d8cd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:54.934761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:54.936622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:54.937448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:54.937448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:54.937448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:54.937448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:54.937448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:54.937448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:54.937448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:54.937448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:54.937448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:54.937448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:54.937448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:54.937448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:54.940651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 236/491] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:19:54.943561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ea5f5-4fda-476c-9f46-4cd28690fa76 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:54.944273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ea5f5-4fda-476c-9f46-4cd28690fa76 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:54.944872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ea5f5-4fda-476c-9f46-4cd28690fa76 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:54.947067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 237/492] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:54.948076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ea5f5-4fda-476c-9f46-4cd28690fa76 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:54.992669 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013dcea-5431-49a2-97d7-99f14621a2ce None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:55.011380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:55.041703 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae992d9-1de6-4ec5-95d3-854b670206ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:55.043225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae992d9-1de6-4ec5-95d3-854b670206ba None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.043710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae992d9-1de6-4ec5-95d3-854b670206ba None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.044177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae992d9-1de6-4ec5-95d3-854b670206ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.047843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:55.085051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ea5f5-4fda-476c-9f46-4cd28690fa76 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:55.135118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559ca9f-268c-46c4-bcbc-f40f85d94728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:55.136713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559ca9f-268c-46c4-bcbc-f40f85d94728 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.137458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559ca9f-268c-46c4-bcbc-f40f85d94728 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.138224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559ca9f-268c-46c4-bcbc-f40f85d94728 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.140012 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559ca9f-268c-46c4-bcbc-f40f85d94728 None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:55.140900 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559ca9f-268c-46c4-bcbc-f40f85d94728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:55.143397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559ca9f-268c-46c4-bcbc-f40f85d94728 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:55.155557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae992d9-1de6-4ec5-95d3-854b670206ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:55.162649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae992d9-1de6-4ec5-95d3-854b670206ba None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:55.162866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 238/493] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:55.162866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73f692-cb04-40f6-98f0-c693538e8e2f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.162866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73f692-cb04-40f6-98f0-c693538e8e2f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.162866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73f692-cb04-40f6-98f0-c693538e8e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.186737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ad73f692-cb04-40f6-98f0-c693538e8e2f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:55.188308 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ad73f692-cb04-40f6-98f0-c693538e8e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:55.278090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 239/494] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:55.280059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2ccd-0b77-4194-8541-283c79417d52 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.280348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2ccd-0b77-4194-8541-283c79417d52 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.280669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2ccd-0b77-4194-8541-283c79417d52 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.288202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 240/495] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:55.288202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b2ccd-0b77-4194-8541-283c79417d52 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.310956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 256/496] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:55.311936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3428ec-4533-4300-9256-0dd6d7b5450c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.312296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3428ec-4533-4300-9256-0dd6d7b5450c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.312613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3428ec-4533-4300-9256-0dd6d7b5450c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.333505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5b3428ec-4533-4300-9256-0dd6d7b5450c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:55.335146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5b3428ec-4533-4300-9256-0dd6d7b5450c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:55.349727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:55.382039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:55.431866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 257/497] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:55.431866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3428ec-4533-4300-9256-0dd6d7b5450c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.439856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b2ccd-0b77-4194-8541-283c79417d52 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:55.478463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a8166-dc13-473a-ac49-acd36c5dabf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:55.479274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a8166-dc13-473a-ac49-acd36c5dabf2 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.479555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a8166-dc13-473a-ac49-acd36c5dabf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.479832 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a8166-dc13-473a-ac49-acd36c5dabf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.491948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a8166-dc13-473a-ac49-acd36c5dabf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:55.493477 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a8166-dc13-473a-ac49-acd36c5dabf2 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-408a8166-dc13-473a-ac49-acd36c5dabf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:55.515886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:55.517858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:55.519571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:55.519571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:55.519571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:55.519571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:55.519571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 241/498] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:55.520165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a8166-dc13-473a-ac49-acd36c5dabf2 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.579267 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3428ec-4533-4300-9256-0dd6d7b5450c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:55.626296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e43fdb-0b29-403f-8587-5f3b41d98fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:55.627093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e43fdb-0b29-403f-8587-5f3b41d98fbb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.627439 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e43fdb-0b29-403f-8587-5f3b41d98fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.627822 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e43fdb-0b29-403f-8587-5f3b41d98fbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.642409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a8166-dc13-473a-ac49-acd36c5dabf2 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:55.643262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e43fdb-0b29-403f-8587-5f3b41d98fbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:55.645845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e43fdb-0b29-403f-8587-5f3b41d98fbb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:55.657664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:55.674026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-f1e43fdb-0b29-403f-8587-5f3b41d98fbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:55.674240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:55.675062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:55.675924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:55.675924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:55.675924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:55.675924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:55.675924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 258/499] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:55.676842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e43fdb-0b29-403f-8587-5f3b41d98fbb None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.700455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af582c21-783d-4f9b-9ad0-57076da8269f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:55.701766 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af582c21-783d-4f9b-9ad0-57076da8269f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.702078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af582c21-783d-4f9b-9ad0-57076da8269f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.702363 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af582c21-783d-4f9b-9ad0-57076da8269f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.713825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af582c21-783d-4f9b-9ad0-57076da8269f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-af582c21-783d-4f9b-9ad0-57076da8269f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:55.722376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:55.724215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 242/500] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:55.726094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511e611-10af-401f-ae23-68aca97aa36d None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.727739 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511e611-10af-401f-ae23-68aca97aa36d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.727739 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511e611-10af-401f-ae23-68aca97aa36d None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.730677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 243/501] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:19:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:19:55.730677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511e611-10af-401f-ae23-68aca97aa36d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.828823 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e43fdb-0b29-403f-8587-5f3b41d98fbb None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:55.847973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511e611-10af-401f-ae23-68aca97aa36d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:55.881216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378bd7f-9076-4720-bd1b-954cef63c280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:55.882064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378bd7f-9076-4720-bd1b-954cef63c280 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.882427 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378bd7f-9076-4720-bd1b-954cef63c280 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.882782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378bd7f-9076-4720-bd1b-954cef63c280 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.894952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378bd7f-9076-4720-bd1b-954cef63c280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:19:55.898462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fff7a-3d90-4ea6-bd87-eb3762e7a588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:55.899615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fff7a-3d90-4ea6-bd87-eb3762e7a588 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.900134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fff7a-3d90-4ea6-bd87-eb3762e7a588 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.900684 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fff7a-3d90-4ea6-bd87-eb3762e7a588 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.902471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fff7a-3d90-4ea6-bd87-eb3762e7a588 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:55.903036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fff7a-3d90-4ea6-bd87-eb3762e7a588 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-1378bd7f-9076-4720-bd1b-954cef63c280 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:19:55.905833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:55.906753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:55.907561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:55.907561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:55.907561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:55.907561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:55.907561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:19:55.907561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:19:55.907561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:19:55.907561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:19:55.907561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:19:55.907561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:19:55.907561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:19:55.907561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:55.908255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fff7a-3d90-4ea6-bd87-eb3762e7a588 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:55.908334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 259/502] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:55.908334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378bd7f-9076-4720-bd1b-954cef63c280 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:55.929191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 244/503] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:55.930784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d910ae-e5f6-4802-9795-d00d063eb259 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:55.931267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d910ae-e5f6-4802-9795-d00d063eb259 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:55.931737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d910ae-e5f6-4802-9795-d00d063eb259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:55.950068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-83d910ae-e5f6-4802-9795-d00d063eb259 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:19:55.951525 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-83d910ae-e5f6-4802-9795-d00d063eb259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:19:55.986414 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:55.994419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:56.033221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378bd7f-9076-4720-bd1b-954cef63c280 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.050571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 245/504] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:19:56.051471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d910ae-e5f6-4802-9795-d00d063eb259 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.073097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabd141-a44d-4f58-b468-9dfb6c81224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.073775 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabd141-a44d-4f58-b468-9dfb6c81224c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.074058 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabd141-a44d-4f58-b468-9dfb6c81224c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.074323 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabd141-a44d-4f58-b468-9dfb6c81224c None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.075069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabd141-a44d-4f58-b468-9dfb6c81224c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:56.075379 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabd141-a44d-4f58-b468-9dfb6c81224c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.078591 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabd141-a44d-4f58-b468-9dfb6c81224c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.090901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 260/505] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:56.093264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabd141-a44d-4f58-b468-9dfb6c81224c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.197384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d910ae-e5f6-4802-9795-d00d063eb259 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.205851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabd141-a44d-4f58-b468-9dfb6c81224c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.225442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:56.238265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b45d7-b785-4e4e-a9da-6be6c09f70d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.238949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b45d7-b785-4e4e-a9da-6be6c09f70d5 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.239245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b45d7-b785-4e4e-a9da-6be6c09f70d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.239522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b45d7-b785-4e4e-a9da-6be6c09f70d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.240294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b45d7-b785-4e4e-a9da-6be6c09f70d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:56.240639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b45d7-b785-4e4e-a9da-6be6c09f70d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.242048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b45d7-b785-4e4e-a9da-6be6c09f70d5 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.247018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1e820-4eaa-4742-82df-5ffd9f6ad65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.249259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1e820-4eaa-4742-82df-5ffd9f6ad65f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.249675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1e820-4eaa-4742-82df-5ffd9f6ad65f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.250046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1e820-4eaa-4742-82df-5ffd9f6ad65f None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.250921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1e820-4eaa-4742-82df-5ffd9f6ad65f None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:56.251319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1e820-4eaa-4742-82df-5ffd9f6ad65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.251471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 246/506] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:56.252319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b45d7-b785-4e4e-a9da-6be6c09f70d5 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.253412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1e820-4eaa-4742-82df-5ffd9f6ad65f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.264987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 261/507] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:56.265869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1e820-4eaa-4742-82df-5ffd9f6ad65f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.293489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:56.376069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b45d7-b785-4e4e-a9da-6be6c09f70d5 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.382510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1e820-4eaa-4742-82df-5ffd9f6ad65f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.415324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6475834-98fa-40da-b95d-65178f9a5e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.416431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6475834-98fa-40da-b95d-65178f9a5e86 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.416934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6475834-98fa-40da-b95d-65178f9a5e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.417337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6475834-98fa-40da-b95d-65178f9a5e86 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.426590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b395763-c090-4e4b-88df-7376d152f85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.427359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6475834-98fa-40da-b95d-65178f9a5e86 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.427509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b395763-c090-4e4b-88df-7376d152f85c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.427703 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b395763-c090-4e4b-88df-7376d152f85c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.427941 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b395763-c090-4e4b-88df-7376d152f85c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.428886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6475834-98fa-40da-b95d-65178f9a5e86 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.442904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b395763-c090-4e4b-88df-7376d152f85c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.444643 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b395763-c090-4e4b-88df-7376d152f85c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-d6475834-98fa-40da-b95d-65178f9a5e86 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:56.446674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:56.447635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:56.448588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:56.448588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:56.448588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:56.448588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:56.448588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:56.448588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:56.448588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:56.448588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:56.448588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:19:56.448588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:56.448588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 247/508] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:56.449174 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6475834-98fa-40da-b95d-65178f9a5e86 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-6b395763-c090-4e4b-88df-7376d152f85c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:56.469432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:56.470677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:56.471713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:56.471713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:56.471713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:56.471713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:56.471713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 262/509] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:56.471713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b395763-c090-4e4b-88df-7376d152f85c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.506976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:56.556635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:56.568337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6475834-98fa-40da-b95d-65178f9a5e86 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.583199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b395763-c090-4e4b-88df-7376d152f85c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.607451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366052ec-e8e1-4f1c-95b6-650c8fd3e9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.609212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366052ec-e8e1-4f1c-95b6-650c8fd3e9fd None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.609656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366052ec-e8e1-4f1c-95b6-650c8fd3e9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.610003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366052ec-e8e1-4f1c-95b6-650c8fd3e9fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.618720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366052ec-e8e1-4f1c-95b6-650c8fd3e9fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.620503 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366052ec-e8e1-4f1c-95b6-650c8fd3e9fd None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.620709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53674e60-9130-453f-8ce7-a7eb4c0d2fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.621521 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53674e60-9130-453f-8ce7-a7eb4c0d2fca None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.621809 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53674e60-9130-453f-8ce7-a7eb4c0d2fca None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.622088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53674e60-9130-453f-8ce7-a7eb4c0d2fca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.634675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53674e60-9130-453f-8ce7-a7eb4c0d2fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.636159 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53674e60-9130-453f-8ce7-a7eb4c0d2fca None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-366052ec-e8e1-4f1c-95b6-650c8fd3e9fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:56.636937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:56.639109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:56.641410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:56.641410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:56.641410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:56.641410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:56.641410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:56.641410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:56.641410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:56.641410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:56.641410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 248/510] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:56.641410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366052ec-e8e1-4f1c-95b6-650c8fd3e9fd None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-53674e60-9130-453f-8ce7-a7eb4c0d2fca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:56.661637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:56.663215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:56.664422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:56.664422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:56.664422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:56.664422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:56.665579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 263/511] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:56.667169 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53674e60-9130-453f-8ce7-a7eb4c0d2fca None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.755149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366052ec-e8e1-4f1c-95b6-650c8fd3e9fd None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.794810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c4b63-3d56-4307-a8e8-18f6f341916b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.795364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c4b63-3d56-4307-a8e8-18f6f341916b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.796122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c4b63-3d56-4307-a8e8-18f6f341916b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.796122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c4b63-3d56-4307-a8e8-18f6f341916b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.807800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:56.808011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c4b63-3d56-4307-a8e8-18f6f341916b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.809556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c4b63-3d56-4307-a8e8-18f6f341916b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-a19c4b63-3d56-4307-a8e8-18f6f341916b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:56.829901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:56.830845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:56.831699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:56.831699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:56.831699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:56.831699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:56.831699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 249/512] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:56.831957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c4b63-3d56-4307-a8e8-18f6f341916b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.836050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:56.836494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53674e60-9130-453f-8ce7-a7eb4c0d2fca None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:56.875796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4648e4-f2a8-4b65-a30e-9b37f0a7b24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:56.876505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4648e4-f2a8-4b65-a30e-9b37f0a7b24a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:56.876796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4648e4-f2a8-4b65-a30e-9b37f0a7b24a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:56.877075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4648e4-f2a8-4b65-a30e-9b37f0a7b24a None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:56.877937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4648e4-f2a8-4b65-a30e-9b37f0a7b24a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:56.878307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4648e4-f2a8-4b65-a30e-9b37f0a7b24a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:56.879739 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4648e4-f2a8-4b65-a30e-9b37f0a7b24a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:56.899985 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 264/513] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:56.900970 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4648e4-f2a8-4b65-a30e-9b37f0a7b24a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:56.958974 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c4b63-3d56-4307-a8e8-18f6f341916b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.014316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c63af-a1ca-4941-8496-93b06b04ac66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.014886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c63af-a1ca-4941-8496-93b06b04ac66 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.015164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c63af-a1ca-4941-8496-93b06b04ac66 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.015438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c63af-a1ca-4941-8496-93b06b04ac66 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.016182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c63af-a1ca-4941-8496-93b06b04ac66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:57.016528 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c63af-a1ca-4941-8496-93b06b04ac66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.017931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c63af-a1ca-4941-8496-93b06b04ac66 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.026670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4648e4-f2a8-4b65-a30e-9b37f0a7b24a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.031183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 250/514] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:57.032054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c63af-a1ca-4941-8496-93b06b04ac66 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.071207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a4941-be99-4432-8293-a93f1ef9ae50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.071577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a4941-be99-4432-8293-a93f1ef9ae50 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.071861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a4941-be99-4432-8293-a93f1ef9ae50 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.072138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a4941-be99-4432-8293-a93f1ef9ae50 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.072978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a4941-be99-4432-8293-a93f1ef9ae50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:57.073295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a4941-be99-4432-8293-a93f1ef9ae50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.074766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a4941-be99-4432-8293-a93f1ef9ae50 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.097129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 265/515] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:57.098246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a4941-be99-4432-8293-a93f1ef9ae50 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.102231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:57.117490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:57.165024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c63af-a1ca-4941-8496-93b06b04ac66 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.203939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545cd86-99a2-4ddc-be56-551994d32b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.204650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545cd86-99a2-4ddc-be56-551994d32b8f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.204941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545cd86-99a2-4ddc-be56-551994d32b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.205219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545cd86-99a2-4ddc-be56-551994d32b8f None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.205976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545cd86-99a2-4ddc-be56-551994d32b8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:57.206290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545cd86-99a2-4ddc-be56-551994d32b8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.207696 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545cd86-99a2-4ddc-be56-551994d32b8f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.221475 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 251/516] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:57.222352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545cd86-99a2-4ddc-be56-551994d32b8f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.287976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a4941-be99-4432-8293-a93f1ef9ae50 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.329091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb92222-5a62-447c-a5eb-c917c4e6fb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.329768 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb92222-5a62-447c-a5eb-c917c4e6fb4b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.330053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb92222-5a62-447c-a5eb-c917c4e6fb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.330328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb92222-5a62-447c-a5eb-c917c4e6fb4b None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.331069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb92222-5a62-447c-a5eb-c917c4e6fb4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:57.331380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb92222-5a62-447c-a5eb-c917c4e6fb4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.332812 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb92222-5a62-447c-a5eb-c917c4e6fb4b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.346619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 266/517] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:57.347206 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb92222-5a62-447c-a5eb-c917c4e6fb4b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.356529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545cd86-99a2-4ddc-be56-551994d32b8f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.382866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:57.397093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f800f4e-23be-45ff-af00-bb30fd89d86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.397753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f800f4e-23be-45ff-af00-bb30fd89d86b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.398037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f800f4e-23be-45ff-af00-bb30fd89d86b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.398314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f800f4e-23be-45ff-af00-bb30fd89d86b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.424828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:57.474255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f800f4e-23be-45ff-af00-bb30fd89d86b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.475736 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f800f4e-23be-45ff-af00-bb30fd89d86b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.476282 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb92222-5a62-447c-a5eb-c917c4e6fb4b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.515494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac51e7-1741-4ff3-ba87-c1dc99e365e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.516172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac51e7-1741-4ff3-ba87-c1dc99e365e3 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.516456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac51e7-1741-4ff3-ba87-c1dc99e365e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.516769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac51e7-1741-4ff3-ba87-c1dc99e365e3 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.517495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac51e7-1741-4ff3-ba87-c1dc99e365e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.518971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac51e7-1741-4ff3-ba87-c1dc99e365e3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.541767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 267/518] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:19:57 2022] POST /identity/v3/users => generated 247 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:19:57.542501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac51e7-1741-4ff3-ba87-c1dc99e365e3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.623513 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 252/519] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:57.624564 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f800f4e-23be-45ff-af00-bb30fd89d86b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.655418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:57.677285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac51e7-1741-4ff3-ba87-c1dc99e365e3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.704461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:57.727369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223ed34-4223-47d1-8c63-dda8c0f2b572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.728041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223ed34-4223-47d1-8c63-dda8c0f2b572 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.728345 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223ed34-4223-47d1-8c63-dda8c0f2b572 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.728646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223ed34-4223-47d1-8c63-dda8c0f2b572 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.729493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223ed34-4223-47d1-8c63-dda8c0f2b572 None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=c77f6ccc7cb84c04ac532007a04a0f5e) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:57.729807 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223ed34-4223-47d1-8c63-dda8c0f2b572 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.731298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223ed34-4223-47d1-8c63-dda8c0f2b572 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.738202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 268/520] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/role_assignments?user.id=c77f6ccc7cb84c04ac532007a04a0f5e&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:57.739072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223ed34-4223-47d1-8c63-dda8c0f2b572 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.800297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f800f4e-23be-45ff-af00-bb30fd89d86b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.864452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dc3ef-49c7-4772-9750-bb3be3eee7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.865734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dc3ef-49c7-4772-9750-bb3be3eee7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.866079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dc3ef-49c7-4772-9750-bb3be3eee7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.866349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dc3ef-49c7-4772-9750-bb3be3eee7b2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.877645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421dc3ef-49c7-4772-9750-bb3be3eee7b2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.879247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421dc3ef-49c7-4772-9750-bb3be3eee7b2 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-421dc3ef-49c7-4772-9750-bb3be3eee7b2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:57.898577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:57.899553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:57.900463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:57.900463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:57.900463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:57.900463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:57.900463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:57.900463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:57.900463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:57.900463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:57.900463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:19:57.900463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:57.900463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 253/521] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:57.901151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dc3ef-49c7-4772-9750-bb3be3eee7b2 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:57.906449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223ed34-4223-47d1-8c63-dda8c0f2b572 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:57.975709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc87f72-41fe-4bc1-8b42-eea69fb143c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:57.976868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc87f72-41fe-4bc1-8b42-eea69fb143c4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:57.977159 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc87f72-41fe-4bc1-8b42-eea69fb143c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:57.977439 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc87f72-41fe-4bc1-8b42-eea69fb143c4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:57.992296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc87f72-41fe-4bc1-8b42-eea69fb143c4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:57.995179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc87f72-41fe-4bc1-8b42-eea69fb143c4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.019330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ffc87f72-41fe-4bc1-8b42-eea69fb143c4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:19:58.027546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:58.029145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:58.030263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:58.030263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:58.030263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:19:58.030263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:19:58.030263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:19:58.030263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:19:58.030263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:19:58.030263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:19:58.030263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:19:58.030263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:58.036674 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 269/522] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:58.038399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc87f72-41fe-4bc1-8b42-eea69fb143c4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.047397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dc3ef-49c7-4772-9750-bb3be3eee7b2 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.106495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3689b-d3e8-4076-aad5-8e3313453a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.108303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3689b-d3e8-4076-aad5-8e3313453a22 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.108828 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3689b-d3e8-4076-aad5-8e3313453a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.109296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3689b-d3e8-4076-aad5-8e3313453a22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.114585 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:58.129335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3689b-d3e8-4076-aad5-8e3313453a22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.131362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3689b-d3e8-4076-aad5-8e3313453a22 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-1dd3689b-d3e8-4076-aad5-8e3313453a22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:58.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:58.160991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:58.163077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:58.163077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:58.163077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:58.163077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:58.163077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 254/523] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:58.163077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3689b-d3e8-4076-aad5-8e3313453a22 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.199047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc87f72-41fe-4bc1-8b42-eea69fb143c4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.243492 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d8520-0d67-4434-ab31-0c7a332b6802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.244300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d8520-0d67-4434-ab31-0c7a332b6802 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.244625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d8520-0d67-4434-ab31-0c7a332b6802 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.244909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d8520-0d67-4434-ab31-0c7a332b6802 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.245778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d8520-0d67-4434-ab31-0c7a332b6802 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:58.246091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d8520-0d67-4434-ab31-0c7a332b6802 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.247670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d8520-0d67-4434-ab31-0c7a332b6802 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.274539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 270/524] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:58.275483 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d8520-0d67-4434-ab31-0c7a332b6802 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.352835 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3689b-d3e8-4076-aad5-8e3313453a22 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.393643 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:58.398361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d8520-0d67-4434-ab31-0c7a332b6802 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.422573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af40b9d-9d3e-4d27-aaa9-6e8127ee9520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.423942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af40b9d-9d3e-4d27-aaa9-6e8127ee9520 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.424648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af40b9d-9d3e-4d27-aaa9-6e8127ee9520 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.425529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af40b9d-9d3e-4d27-aaa9-6e8127ee9520 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.427303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af40b9d-9d3e-4d27-aaa9-6e8127ee9520 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:58.428024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af40b9d-9d3e-4d27-aaa9-6e8127ee9520 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.431285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af40b9d-9d3e-4d27-aaa9-6e8127ee9520 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.441067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:58.446558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e84e3-35d6-4bd5-a4cf-5565990365ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.447942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e84e3-35d6-4bd5-a4cf-5565990365ef None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.448609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e84e3-35d6-4bd5-a4cf-5565990365ef None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.449241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e84e3-35d6-4bd5-a4cf-5565990365ef None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.451062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e84e3-35d6-4bd5-a4cf-5565990365ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:58.451724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e84e3-35d6-4bd5-a4cf-5565990365ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.454781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e84e3-35d6-4bd5-a4cf-5565990365ef None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.484200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 271/525] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:58.485494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e84e3-35d6-4bd5-a4cf-5565990365ef None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.496789 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 255/526] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:58.498751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af40b9d-9d3e-4d27-aaa9-6e8127ee9520 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.618742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e84e3-35d6-4bd5-a4cf-5565990365ef None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.661266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b68641-bc87-406b-b363-ae1eaae70cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.661868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b68641-bc87-406b-b363-ae1eaae70cbd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.662157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b68641-bc87-406b-b363-ae1eaae70cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.662440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b68641-bc87-406b-b363-ae1eaae70cbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.695324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:58.708563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af40b9d-9d3e-4d27-aaa9-6e8127ee9520 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.756616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b68641-bc87-406b-b363-ae1eaae70cbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.758104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b68641-bc87-406b-b363-ae1eaae70cbd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.767590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb18bc-3649-4c06-abcd-22b9de9ca323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:58.769441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb18bc-3649-4c06-abcd-22b9de9ca323 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:58.769730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb18bc-3649-4c06-abcd-22b9de9ca323 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:58.770013 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb18bc-3649-4c06-abcd-22b9de9ca323 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:58.782774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb18bc-3649-4c06-abcd-22b9de9ca323 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:58.784759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb18bc-3649-4c06-abcd-22b9de9ca323 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-15fb18bc-3649-4c06-abcd-22b9de9ca323 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:19:58.805681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:58.806589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:58.807438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:19:58.807438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:19:58.807438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:19:58.807438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:19:58.807438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:19:58.807438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:19:58.807438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:19:58.807438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:19:58.816652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 256/527] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:58.817843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb18bc-3649-4c06-abcd-22b9de9ca323 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.894697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 272/528] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:58.894697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b68641-bc87-406b-b363-ae1eaae70cbd None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:58.896151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:58.961979 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb18bc-3649-4c06-abcd-22b9de9ca323 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:58.995794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:59.009634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eabb6f-bd45-494f-a389-5a7d9218901d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:59.010933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eabb6f-bd45-494f-a389-5a7d9218901d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:59.012460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eabb6f-bd45-494f-a389-5a7d9218901d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:59.013139 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eabb6f-bd45-494f-a389-5a7d9218901d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:59.049333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b68641-bc87-406b-b363-ae1eaae70cbd None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:59.091714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9916f-2d28-4a00-ac55-c200ee3a3bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:59.092744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9916f-2d28-4a00-ac55-c200ee3a3bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:59.093047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9916f-2d28-4a00-ac55-c200ee3a3bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:59.093327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e9916f-2d28-4a00-ac55-c200ee3a3bc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:59.094174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e9916f-2d28-4a00-ac55-c200ee3a3bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:59.094492 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e9916f-2d28-4a00-ac55-c200ee3a3bc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:59.098387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e9916f-2d28-4a00-ac55-c200ee3a3bc6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:59.123315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 273/529] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:59.123315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9916f-2d28-4a00-ac55-c200ee3a3bc6 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:59.132339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eabb6f-bd45-494f-a389-5a7d9218901d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:59.135436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eabb6f-bd45-494f-a389-5a7d9218901d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:59.203399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:59.258450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e9916f-2d28-4a00-ac55-c200ee3a3bc6 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:59.262477 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 257/530] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:19:59.263540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eabb6f-bd45-494f-a389-5a7d9218901d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:59.285908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:59.304012 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0329a-a2d4-47f6-a908-827e71c7c753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:59.305252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0329a-a2d4-47f6-a908-827e71c7c753 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:59.305745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0329a-a2d4-47f6-a908-827e71c7c753 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:59.306401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0329a-a2d4-47f6-a908-827e71c7c753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:59.319511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0329a-a2d4-47f6-a908-827e71c7c753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:59.321405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0329a-a2d4-47f6-a908-827e71c7c753 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e6d0329a-a2d4-47f6-a908-827e71c7c753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:59.341937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:59.342908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:59.343781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:59.343781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:59.343781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:59.343781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:59.346316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 274/531] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:59.351808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0329a-a2d4-47f6-a908-827e71c7c753 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:59.460217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eabb6f-bd45-494f-a389-5a7d9218901d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:59.473004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0329a-a2d4-47f6-a908-827e71c7c753 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:59.534926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67e419-4466-4f55-b17e-255991869240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:59.536237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67e419-4466-4f55-b17e-255991869240 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:59.536833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67e419-4466-4f55-b17e-255991869240 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:59.537347 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67e419-4466-4f55-b17e-255991869240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:59.538049 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3534f1e-4e51-4bcb-b2a8-992de23ac461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:19:59.539411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3534f1e-4e51-4bcb-b2a8-992de23ac461 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:19:59.539914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3534f1e-4e51-4bcb-b2a8-992de23ac461 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:19:59.540378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3534f1e-4e51-4bcb-b2a8-992de23ac461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:19:59.546813 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3534f1e-4e51-4bcb-b2a8-992de23ac461 None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=78b67db080874f3ebb508239c7a8ccf8) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:19:59.547378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3534f1e-4e51-4bcb-b2a8-992de23ac461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:59.566876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3534f1e-4e51-4bcb-b2a8-992de23ac461 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:59.569882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67e419-4466-4f55-b17e-255991869240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:19:59.569882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67e419-4466-4f55-b17e-255991869240 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:19:59.598028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 258/532] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:19:59 2022] GET /identity/v3/role_assignments?user.id=78b67db080874f3ebb508239c7a8ccf8&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 259 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:19:59.598028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3534f1e-4e51-4bcb-b2a8-992de23ac461 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-4f67e419-4466-4f55-b17e-255991869240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:19:59.639293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:19:59.640225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:19:59.641106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:19:59.641106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:19:59.641106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:19:59.641106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:19:59.643344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 275/533] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:19:59.648589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:59.652601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67e419-4466-4f55-b17e-255991869240 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:19:59.725334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:19:59.885436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3534f1e-4e51-4bcb-b2a8-992de23ac461 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:19:59.980568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67e419-4466-4f55-b17e-255991869240 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:00.042580 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c1bd0-9802-4c12-8ee8-57b3732683ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:00.044233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c1bd0-9802-4c12-8ee8-57b3732683ce None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.044995 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c1bd0-9802-4c12-8ee8-57b3732683ce None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.045726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c1bd0-9802-4c12-8ee8-57b3732683ce None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.056386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c1bd0-9802-4c12-8ee8-57b3732683ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:00.057734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c1bd0-9802-4c12-8ee8-57b3732683ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:00.068508 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c1bd0-9802-4c12-8ee8-57b3732683ce None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:00.073984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce67cd-3a73-4b8b-9ac6-6f49146a9e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:00.074962 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce67cd-3a73-4b8b-9ac6-6f49146a9e21 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.076054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce67cd-3a73-4b8b-9ac6-6f49146a9e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.076519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce67cd-3a73-4b8b-9ac6-6f49146a9e21 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.077444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce67cd-3a73-4b8b-9ac6-6f49146a9e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:00.077940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce67cd-3a73-4b8b-9ac6-6f49146a9e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:00.079541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce67cd-3a73-4b8b-9ac6-6f49146a9e21 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:00.088503 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 259/534] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 492 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:00.095190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c1bd0-9802-4c12-8ee8-57b3732683ce None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:00.156019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 276/535] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 503 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:00.170618 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce67cd-3a73-4b8b-9ac6-6f49146a9e21 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:00.389358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:00.470926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c1bd0-9802-4c12-8ee8-57b3732683ce None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:00.490760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:00.513907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce67cd-3a73-4b8b-9ac6-6f49146a9e21 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:00.557422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02dd01-b704-4315-9495-a6f8cfdbc1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:00.559318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02dd01-b704-4315-9495-a6f8cfdbc1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.561323 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02dd01-b704-4315-9495-a6f8cfdbc1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.561810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02dd01-b704-4315-9495-a6f8cfdbc1f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.574352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02dd01-b704-4315-9495-a6f8cfdbc1f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:00.578269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02dd01-b704-4315-9495-a6f8cfdbc1f1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:00.584401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978a6e9-5184-46d8-9402-421a3e7555f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:00.585485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978a6e9-5184-46d8-9402-421a3e7555f1 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.585935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978a6e9-5184-46d8-9402-421a3e7555f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.586372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978a6e9-5184-46d8-9402-421a3e7555f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-8f02dd01-b704-4315-9495-a6f8cfdbc1f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:00.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:00.611429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:00.612459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:00.612459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:00.612459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:00.612459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:00.612459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:00.612459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:00.612459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:00.612459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:00.614124 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978a6e9-5184-46d8-9402-421a3e7555f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:00.614400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 277/536] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:00.616173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978a6e9-5184-46d8-9402-421a3e7555f1 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:00.623371 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02dd01-b704-4315-9495-a6f8cfdbc1f1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-e978a6e9-5184-46d8-9402-421a3e7555f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:00.634620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:00.635553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:00.636404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:00.636404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:00.636404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:00.636404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:00.636404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:00.636404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:00.636404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:00.636404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:00.640392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 260/537] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 550 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:00.650034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978a6e9-5184-46d8-9402-421a3e7555f1 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:00.804575 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02dd01-b704-4315-9495-a6f8cfdbc1f1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:00.828496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978a6e9-5184-46d8-9402-421a3e7555f1 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:00.855027 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68347c48-45e7-45f6-bbec-533bc9116811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:00.856102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68347c48-45e7-45f6-bbec-533bc9116811 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.857234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68347c48-45e7-45f6-bbec-533bc9116811 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.857919 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68347c48-45e7-45f6-bbec-533bc9116811 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.859026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68347c48-45e7-45f6-bbec-533bc9116811 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:00.859588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68347c48-45e7-45f6-bbec-533bc9116811 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:00.861623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68347c48-45e7-45f6-bbec-533bc9116811 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:00.876543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e03fb-83d0-4fd0-8465-0cd24cb1fea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:00.877265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e03fb-83d0-4fd0-8465-0cd24cb1fea6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:00.877552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e03fb-83d0-4fd0-8465-0cd24cb1fea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:00.877826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e03fb-83d0-4fd0-8465-0cd24cb1fea6 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:00.878642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e03fb-83d0-4fd0-8465-0cd24cb1fea6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:00.878950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e03fb-83d0-4fd0-8465-0cd24cb1fea6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:00.880421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e03fb-83d0-4fd0-8465-0cd24cb1fea6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:00.901027 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:00.904872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 261/538] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:00.907063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 278/539] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:00.913091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68347c48-45e7-45f6-bbec-533bc9116811 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:00.913297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e03fb-83d0-4fd0-8465-0cd24cb1fea6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:00.919924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:01.121808 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e03fb-83d0-4fd0-8465-0cd24cb1fea6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:01.160967 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d10726-b01f-46bd-ab47-af161742e61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:01.161792 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d10726-b01f-46bd-ab47-af161742e61f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.162169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d10726-b01f-46bd-ab47-af161742e61f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.162308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68347c48-45e7-45f6-bbec-533bc9116811 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:01.162733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d10726-b01f-46bd-ab47-af161742e61f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.163564 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d10726-b01f-46bd-ab47-af161742e61f None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=a697d1e7e42048a8a1fb997bcc7b3fab) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:01.164001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d10726-b01f-46bd-ab47-af161742e61f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:01.165609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d10726-b01f-46bd-ab47-af161742e61f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:01.172597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 262/540] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:00 2022] GET /identity/v3/role_assignments?user.id=a697d1e7e42048a8a1fb997bcc7b3fab&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:01.174251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9429a-e173-4dec-b97a-968dc4f91ba1 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.174643 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9429a-e173-4dec-b97a-968dc4f91ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.175024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9429a-e173-4dec-b97a-968dc4f91ba1 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.176906 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 263/541] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:01.181120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522432-fbdc-4895-b313-9f49480ff342 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.181559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522432-fbdc-4895-b313-9f49480ff342 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.181942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522432-fbdc-4895-b313-9f49480ff342 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.184452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 264/542] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:01.187925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcec2ff-bad4-42a7-a648-aafa5181b1b1 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.187925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcec2ff-bad4-42a7-a648-aafa5181b1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.188149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcec2ff-bad4-42a7-a648-aafa5181b1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.212302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-0dcec2ff-bad4-42a7-a648-aafa5181b1b1 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:01.213975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0dcec2ff-bad4-42a7-a648-aafa5181b1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:01.252939 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d40e4-a152-492d-b5f4-ffff676a9d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:01.254146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d40e4-a152-492d-b5f4-ffff676a9d99 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.254693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d40e4-a152-492d-b5f4-ffff676a9d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.255246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d40e4-a152-492d-b5f4-ffff676a9d99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.257196 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d40e4-a152-492d-b5f4-ffff676a9d99 None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=8a120d9e7e204dc5b972fd9402897b41) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:01.257811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d40e4-a152-492d-b5f4-ffff676a9d99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:01.261212 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d40e4-a152-492d-b5f4-ffff676a9d99 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:01.276270 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 279/543] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:00 2022] GET /identity/v3/role_assignments?user.id=8a120d9e7e204dc5b972fd9402897b41&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 259 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:01.281288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1e154-9bc2-4bbe-9776-6cb2dc3c3938 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.281817 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1e154-9bc2-4bbe-9776-6cb2dc3c3938 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.282351 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1e154-9bc2-4bbe-9776-6cb2dc3c3938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.292257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:01.308671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 265/544] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:01.321480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-30c1e154-9bc2-4bbe-9776-6cb2dc3c3938 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:01.323874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-30c1e154-9bc2-4bbe-9776-6cb2dc3c3938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:01.375317 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcec2ff-bad4-42a7-a648-aafa5181b1b1 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:01.496424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:01.510516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 280/545] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:01.531102 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcec2ff-bad4-42a7-a648-aafa5181b1b1 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:01.536759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1e154-9bc2-4bbe-9776-6cb2dc3c3938 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:01.599974 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5add3-633c-48ec-86c7-9f0715d1db8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:01.600823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5add3-633c-48ec-86c7-9f0715d1db8a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.601107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5add3-633c-48ec-86c7-9f0715d1db8a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.601386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5add3-633c-48ec-86c7-9f0715d1db8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.628758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5add3-633c-48ec-86c7-9f0715d1db8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-b7f5add3-633c-48ec-86c7-9f0715d1db8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:01.637569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:01.640287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:01.642208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 266/546] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:01.649512 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5add3-633c-48ec-86c7-9f0715d1db8a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:01.719185 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:01.797993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1e154-9bc2-4bbe-9776-6cb2dc3c3938 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:01.871345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5add3-633c-48ec-86c7-9f0715d1db8a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:01.873461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e2daa-c990-403b-8bef-94f345aa86f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:01.875174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e2daa-c990-403b-8bef-94f345aa86f0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.875977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e2daa-c990-403b-8bef-94f345aa86f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.876782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e2daa-c990-403b-8bef-94f345aa86f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.922079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e2daa-c990-403b-8bef-94f345aa86f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-a62e2daa-c990-403b-8bef-94f345aa86f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:01.932961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:01.935304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:01.936989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 281/547] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:01.941431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e2daa-c990-403b-8bef-94f345aa86f0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:01.981438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce6091-d291-4e82-9b76-4bf53b18af34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:01.982708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce6091-d291-4e82-9b76-4bf53b18af34 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:01.983006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce6091-d291-4e82-9b76-4bf53b18af34 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:01.983295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce6091-d291-4e82-9b76-4bf53b18af34 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:01.984119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce6091-d291-4e82-9b76-4bf53b18af34 None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:01.984437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce6091-d291-4e82-9b76-4bf53b18af34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:01.985927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce6091-d291-4e82-9b76-4bf53b18af34 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:01.995730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 267/548] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:02.005705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce6091-d291-4e82-9b76-4bf53b18af34 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:02.079971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:02.174045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e2daa-c990-403b-8bef-94f345aa86f0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:02.242255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce6091-d291-4e82-9b76-4bf53b18af34 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:02.246085 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe804831-f703-425e-bce7-f2ce29fc88d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:02.248291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe804831-f703-425e-bce7-f2ce29fc88d6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.249069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe804831-f703-425e-bce7-f2ce29fc88d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.249810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe804831-f703-425e-bce7-f2ce29fc88d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.251542 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe804831-f703-425e-bce7-f2ce29fc88d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:02.252389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe804831-f703-425e-bce7-f2ce29fc88d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:02.255108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe804831-f703-425e-bce7-f2ce29fc88d6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:02.267700 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:02.271437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 282/549] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:02.274531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c8d7d-2285-4970-b933-f63007b9a8ca None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.275375 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c8d7d-2285-4970-b933-f63007b9a8ca None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.276222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c8d7d-2285-4970-b933-f63007b9a8ca None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.280770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 283/550] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:02.282962 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c8d7d-2285-4970-b933-f63007b9a8ca None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:02.314286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd115d-6976-4fca-b827-d248f2cbce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:02.315968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd115d-6976-4fca-b827-d248f2cbce9c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.316198 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd115d-6976-4fca-b827-d248f2cbce9c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.316796 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd115d-6976-4fca-b827-d248f2cbce9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.336596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd115d-6976-4fca-b827-d248f2cbce9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:02.339048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd115d-6976-4fca-b827-d248f2cbce9c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-edfd115d-6976-4fca-b827-d248f2cbce9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:02.371417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:02.373070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:02.373981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:02.373981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:02.373981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:02.373981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:02.375054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 268/551] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:02.377303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5fd48-aa31-4927-b43d-2d3ab0dc431c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.377830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5fd48-aa31-4927-b43d-2d3ab0dc431c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.378505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5fd48-aa31-4927-b43d-2d3ab0dc431c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.407910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-eda5fd48-aa31-4927-b43d-2d3ab0dc431c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:02.410753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-eda5fd48-aa31-4927-b43d-2d3ab0dc431c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:02.454372 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c8d7d-2285-4970-b933-f63007b9a8ca None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:02.479824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:02.517392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9908e-b361-4f53-bdcd-0bce34fb2f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:02.519288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9908e-b361-4f53-bdcd-0bce34fb2f80 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.520032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9908e-b361-4f53-bdcd-0bce34fb2f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.521427 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9908e-b361-4f53-bdcd-0bce34fb2f80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.529580 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 269/552] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:02.531354 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5fd48-aa31-4927-b43d-2d3ab0dc431c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:02.543312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9908e-b361-4f53-bdcd-0bce34fb2f80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:02.546111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9908e-b361-4f53-bdcd-0bce34fb2f80 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-5aa9908e-b361-4f53-bdcd-0bce34fb2f80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:02.585088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:02.586207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:02.587216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:02.587216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:02.587216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:02.587216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:02.589571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 284/553] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:02.592508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9908e-b361-4f53-bdcd-0bce34fb2f80 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:02.697655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:02.706753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5fd48-aa31-4927-b43d-2d3ab0dc431c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:02.744868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630c236-99de-4a6e-beaa-d116112b1c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:02.745581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630c236-99de-4a6e-beaa-d116112b1c36 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.745862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630c236-99de-4a6e-beaa-d116112b1c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.746138 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630c236-99de-4a6e-beaa-d116112b1c36 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.746885 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630c236-99de-4a6e-beaa-d116112b1c36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:02.747753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630c236-99de-4a6e-beaa-d116112b1c36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:02.749488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630c236-99de-4a6e-beaa-d116112b1c36 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:02.766250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9908e-b361-4f53-bdcd-0bce34fb2f80 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:02.768591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 270/554] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:02.769637 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630c236-99de-4a6e-beaa-d116112b1c36 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:02.813341 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c4443-0159-4935-894e-605c9c803a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:02.814207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c4443-0159-4935-894e-605c9c803a58 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.814486 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c4443-0159-4935-894e-605c9c803a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.814842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c4443-0159-4935-894e-605c9c803a58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.827334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c4443-0159-4935-894e-605c9c803a58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-a54c4443-0159-4935-894e-605c9c803a58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:02.835364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:02.838902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:02.842110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 285/555] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:02.854313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c4443-0159-4935-894e-605c9c803a58 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:02.877225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:02.912379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630c236-99de-4a6e-beaa-d116112b1c36 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:02.954655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c4443-0159-4935-894e-605c9c803a58 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:02.962442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3eb6c-c333-4a23-9a1a-8bd035f6637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:02.964002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3eb6c-c333-4a23-9a1a-8bd035f6637e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:02.964451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3eb6c-c333-4a23-9a1a-8bd035f6637e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:02.964928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3eb6c-c333-4a23-9a1a-8bd035f6637e None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:02.966419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3eb6c-c333-4a23-9a1a-8bd035f6637e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:02.966896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3eb6c-c333-4a23-9a1a-8bd035f6637e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:02.969293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3eb6c-c333-4a23-9a1a-8bd035f6637e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:02.984797 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 271/556] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:02.985935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3eb6c-c333-4a23-9a1a-8bd035f6637e None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:03.005026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:03.010517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ef5ef-10b1-4ef8-8790-aca3afc2128b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:03.011275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ef5ef-10b1-4ef8-8790-aca3afc2128b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.011547 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ef5ef-10b1-4ef8-8790-aca3afc2128b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.011825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ef5ef-10b1-4ef8-8790-aca3afc2128b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.033967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ef5ef-10b1-4ef8-8790-aca3afc2128b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:03.038719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ef5ef-10b1-4ef8-8790-aca3afc2128b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e38ef5ef-10b1-4ef8-8790-aca3afc2128b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:03.058552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:03.060467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:03.062795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:03.062795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:03.062795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:03.062795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:03.062795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:03.062795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:03.062795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:03.062795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:03.062795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:03.062795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:03.065011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 286/557] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:03.066245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ef5ef-10b1-4ef8-8790-aca3afc2128b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:03.148573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3eb6c-c333-4a23-9a1a-8bd035f6637e None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:03.218280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:03.220203 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a6f4b-2690-4f0b-822d-29fda5f6c99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:03.221949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a6f4b-2690-4f0b-822d-29fda5f6c99c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.223258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a6f4b-2690-4f0b-822d-29fda5f6c99c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.224403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a6f4b-2690-4f0b-822d-29fda5f6c99c None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.226272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a6f4b-2690-4f0b-822d-29fda5f6c99c None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:03.227506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a6f4b-2690-4f0b-822d-29fda5f6c99c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:03.230447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ef5ef-10b1-4ef8-8790-aca3afc2128b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:03.231072 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a6f4b-2690-4f0b-822d-29fda5f6c99c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:03.252110 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 272/558] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:03.253009 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a6f4b-2690-4f0b-822d-29fda5f6c99c None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:03.290573 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575e895-b01b-4a26-a509-d5e52e4207b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:03.291601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575e895-b01b-4a26-a509-d5e52e4207b9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.291885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575e895-b01b-4a26-a509-d5e52e4207b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.292153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575e895-b01b-4a26-a509-d5e52e4207b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.312550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575e895-b01b-4a26-a509-d5e52e4207b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:03.317496 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575e895-b01b-4a26-a509-d5e52e4207b9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-6575e895-b01b-4a26-a509-d5e52e4207b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:03.355506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:03.357825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:03.359695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:03.359695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:03.359695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:03.359695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:03.359695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:03.359695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:03.359695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:03.359695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:03.359695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:03.359695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:03.364420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:03.369714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 287/559] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 22:20:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:03.371022 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575e895-b01b-4a26-a509-d5e52e4207b9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:03.411099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a6f4b-2690-4f0b-822d-29fda5f6c99c None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:03.468256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d9d0d-67f9-4ab9-b50d-337c8aee5055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:03.469349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d9d0d-67f9-4ab9-b50d-337c8aee5055 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.469791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d9d0d-67f9-4ab9-b50d-337c8aee5055 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.470217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d9d0d-67f9-4ab9-b50d-337c8aee5055 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.471967 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d9d0d-67f9-4ab9-b50d-337c8aee5055 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:03.472798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d9d0d-67f9-4ab9-b50d-337c8aee5055 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:03.474796 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d9d0d-67f9-4ab9-b50d-337c8aee5055 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:03.501666 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 273/560] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jul 21 22:20:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:03.502763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d9d0d-67f9-4ab9-b50d-337c8aee5055 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:03.544407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575e895-b01b-4a26-a509-d5e52e4207b9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:03.592874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f21c0a-382d-44de-8fc2-11eb4d4be381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:03.594676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f21c0a-382d-44de-8fc2-11eb4d4be381 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.595139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f21c0a-382d-44de-8fc2-11eb4d4be381 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.595574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f21c0a-382d-44de-8fc2-11eb4d4be381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.610980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f21c0a-382d-44de-8fc2-11eb4d4be381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:03.612514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f21c0a-382d-44de-8fc2-11eb4d4be381 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-23f21c0a-382d-44de-8fc2-11eb4d4be381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:03.631871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:03.632957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:03.639456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:03.639456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:03.639456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:03.639456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:03.639456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 288/561] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:03.639456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f21c0a-382d-44de-8fc2-11eb4d4be381 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:03.641899 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:03.713883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d9d0d-67f9-4ab9-b50d-337c8aee5055 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:03.772802 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a9236-4362-4511-ae66-6d0600e374eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:03.773387 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a9236-4362-4511-ae66-6d0600e374eb None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.773686 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a9236-4362-4511-ae66-6d0600e374eb None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.773967 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a9236-4362-4511-ae66-6d0600e374eb None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.774849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a9236-4362-4511-ae66-6d0600e374eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:03.775157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a9236-4362-4511-ae66-6d0600e374eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:03.776659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a9236-4362-4511-ae66-6d0600e374eb None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:03.780855 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:03.809532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 274/562] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 22:20:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:03.811885 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9131-61a0-465c-894a-21d08fa2fb3a None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.812419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9131-61a0-465c-894a-21d08fa2fb3a None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.812954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9131-61a0-465c-894a-21d08fa2fb3a None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.820879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 275/563] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:03.820879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb9131-61a0-465c-894a-21d08fa2fb3a None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:03.826915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f21c0a-382d-44de-8fc2-11eb4d4be381 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:03.885525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e77d23-b8a6-4c38-85ab-6611f37a62d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:03.886381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e77d23-b8a6-4c38-85ab-6611f37a62d8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:03.886780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e77d23-b8a6-4c38-85ab-6611f37a62d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:03.887259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e77d23-b8a6-4c38-85ab-6611f37a62d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:03.986595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb9131-61a0-465c-894a-21d08fa2fb3a None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.018033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e77d23-b8a6-4c38-85ab-6611f37a62d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:04.019752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e77d23-b8a6-4c38-85ab-6611f37a62d8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:04.023462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:04.046799 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071c5b6-477a-40ca-b0c4-7b6158dac551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.047944 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071c5b6-477a-40ca-b0c4-7b6158dac551 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.048418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071c5b6-477a-40ca-b0c4-7b6158dac551 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.048940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071c5b6-477a-40ca-b0c4-7b6158dac551 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.050206 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071c5b6-477a-40ca-b0c4-7b6158dac551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:04.050754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071c5b6-477a-40ca-b0c4-7b6158dac551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:04.053358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071c5b6-477a-40ca-b0c4-7b6158dac551 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:04.075983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 276/564] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:04.077627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071c5b6-477a-40ca-b0c4-7b6158dac551 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.173412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 289/565] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:20:04.175010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad7764-5d62-4274-8912-915d145d9e8b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.175425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad7764-5d62-4274-8912-915d145d9e8b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.175811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad7764-5d62-4274-8912-915d145d9e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.182017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:04.194894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8bad7764-5d62-4274-8912-915d145d9e8b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:04.196134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8bad7764-5d62-4274-8912-915d145d9e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:04.243322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071c5b6-477a-40ca-b0c4-7b6158dac551 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.272587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 290/566] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:04.273822 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad7764-5d62-4274-8912-915d145d9e8b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.298573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6f1e5-d8f5-4744-b2c2-a191427443d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.299875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6f1e5-d8f5-4744-b2c2-a191427443d8 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.300423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6f1e5-d8f5-4744-b2c2-a191427443d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.301014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6f1e5-d8f5-4744-b2c2-a191427443d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.319359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6f1e5-d8f5-4744-b2c2-a191427443d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:04.322020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6f1e5-d8f5-4744-b2c2-a191427443d8 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:04.327434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-aaa6f1e5-d8f5-4744-b2c2-a191427443d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:04.348207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:04.350096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:04.351765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:04.351765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:04.351765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:04.351765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:04.351765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 277/567] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:04.352045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21588137-eb74-4960-acc0-ce1a7fef65cb None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.352600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21588137-eb74-4960-acc0-ce1a7fef65cb None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.352991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21588137-eb74-4960-acc0-ce1a7fef65cb None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.355546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 278/568] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:04.357529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21588137-eb74-4960-acc0-ce1a7fef65cb None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.428471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad7764-5d62-4274-8912-915d145d9e8b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.477669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21588137-eb74-4960-acc0-ce1a7fef65cb None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.481068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:04.488023 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ef47d-00d2-4d96-a506-e92a261965e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.488956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ef47d-00d2-4d96-a506-e92a261965e5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.489239 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ef47d-00d2-4d96-a506-e92a261965e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.489836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ef47d-00d2-4d96-a506-e92a261965e5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.501631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ef47d-00d2-4d96-a506-e92a261965e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:04.504431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ef47d-00d2-4d96-a506-e92a261965e5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:04.522377 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97982f9-1ba2-42ee-bfdb-5ae8b4cee57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-df9ef47d-00d2-4d96-a506-e92a261965e5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:04.522671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:04.524577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:04.526755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:04.526755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:04.526755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:04.526755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:04.526755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:04.526755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:04.526755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:04.526755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:04.526755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:20:04.526755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:04.527923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97982f9-1ba2-42ee-bfdb-5ae8b4cee57f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.528101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 291/569] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:04.528101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ef47d-00d2-4d96-a506-e92a261965e5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.532833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97982f9-1ba2-42ee-bfdb-5ae8b4cee57f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.533187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97982f9-1ba2-42ee-bfdb-5ae8b4cee57f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.544598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97982f9-1ba2-42ee-bfdb-5ae8b4cee57f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:04.546073 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97982f9-1ba2-42ee-bfdb-5ae8b4cee57f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-f97982f9-1ba2-42ee-bfdb-5ae8b4cee57f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:04.579368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:04.580952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:04.582221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:04.582221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:04.582221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:04.582221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:04.582221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 279/570] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:04.583096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97982f9-1ba2-42ee-bfdb-5ae8b4cee57f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.634005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:04.642925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ef47d-00d2-4d96-a506-e92a261965e5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.681143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6a4e8-65b9-45cf-8a25-4247f5e4d6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.682233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6a4e8-65b9-45cf-8a25-4247f5e4d6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.682614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6a4e8-65b9-45cf-8a25-4247f5e4d6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.683088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6a4e8-65b9-45cf-8a25-4247f5e4d6a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.698008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6a4e8-65b9-45cf-8a25-4247f5e4d6a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-22e6a4e8-65b9-45cf-8a25-4247f5e4d6a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:04.707463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:04.709651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 292/571] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:04.711259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e550978-dd3d-4066-ae19-6e2f7583d3a2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.712780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e550978-dd3d-4066-ae19-6e2f7583d3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.712780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e550978-dd3d-4066-ae19-6e2f7583d3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.729813 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97982f9-1ba2-42ee-bfdb-5ae8b4cee57f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.732193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9e550978-dd3d-4066-ae19-6e2f7583d3a2 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:04.734111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9e550978-dd3d-4066-ae19-6e2f7583d3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:04.787433 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff2a4d-a339-4458-bbc1-11e81431fc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.788673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff2a4d-a339-4458-bbc1-11e81431fc9c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.789163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff2a4d-a339-4458-bbc1-11e81431fc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.789640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff2a4d-a339-4458-bbc1-11e81431fc9c None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:04.790959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff2a4d-a339-4458-bbc1-11e81431fc9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:04.791491 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff2a4d-a339-4458-bbc1-11e81431fc9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:04.793986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff2a4d-a339-4458-bbc1-11e81431fc9c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:04.812839 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 280/572] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:04.814372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff2a4d-a339-4458-bbc1-11e81431fc9c None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.839473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:04.865056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 293/573] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:04.865056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e550978-dd3d-4066-ae19-6e2f7583d3a2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:04.956128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff2a4d-a339-4458-bbc1-11e81431fc9c None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:04.985089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:04.998099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fd36f-1e76-4ebf-84f5-72c7612e5af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:04.998925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fd36f-1e76-4ebf-84f5-72c7612e5af8 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:04.999327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fd36f-1e76-4ebf-84f5-72c7612e5af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:04.999798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fd36f-1e76-4ebf-84f5-72c7612e5af8 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.000907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fd36f-1e76-4ebf-84f5-72c7612e5af8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:05.001430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fd36f-1e76-4ebf-84f5-72c7612e5af8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.003063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639fd36f-1e76-4ebf-84f5-72c7612e5af8 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.013619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e550978-dd3d-4066-ae19-6e2f7583d3a2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.032458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 281/574] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jul 21 22:20:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:05.033974 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fd36f-1e76-4ebf-84f5-72c7612e5af8 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.059021 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb2b46-9370-44bd-ba84-68f0a33333e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.060268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb2b46-9370-44bd-ba84-68f0a33333e9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.060268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb2b46-9370-44bd-ba84-68f0a33333e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.060654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb2b46-9370-44bd-ba84-68f0a33333e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.061096 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb2b46-9370-44bd-ba84-68f0a33333e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:05.061408 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb2b46-9370-44bd-ba84-68f0a33333e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.062837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb2b46-9370-44bd-ba84-68f0a33333e9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.076370 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 294/575] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:05.077482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb2b46-9370-44bd-ba84-68f0a33333e9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.125430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:05.157482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fd36f-1e76-4ebf-84f5-72c7612e5af8 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.204254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068ea59-1e66-4f65-b48f-6fd03bb863e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.205414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068ea59-1e66-4f65-b48f-6fd03bb863e5 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.205920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068ea59-1e66-4f65-b48f-6fd03bb863e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.206414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068ea59-1e66-4f65-b48f-6fd03bb863e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.207822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068ea59-1e66-4f65-b48f-6fd03bb863e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:05.208352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068ea59-1e66-4f65-b48f-6fd03bb863e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.209172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb2b46-9370-44bd-ba84-68f0a33333e9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.210953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068ea59-1e66-4f65-b48f-6fd03bb863e5 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.226187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 282/576] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:05.227730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068ea59-1e66-4f65-b48f-6fd03bb863e5 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.261005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df62a58-0b5b-4ddd-8f53-5cbcff4f0f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.262262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df62a58-0b5b-4ddd-8f53-5cbcff4f0f1f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.262735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df62a58-0b5b-4ddd-8f53-5cbcff4f0f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.263184 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df62a58-0b5b-4ddd-8f53-5cbcff4f0f1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.275773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df62a58-0b5b-4ddd-8f53-5cbcff4f0f1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.278702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df62a58-0b5b-4ddd-8f53-5cbcff4f0f1f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-2df62a58-0b5b-4ddd-8f53-5cbcff4f0f1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:05.298113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:05.300092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:05.301848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:05.301848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:05.301848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:05.301848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:05.301848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:05.301848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:05.301848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:05.301848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:05.301848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 295/577] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:05.302783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df62a58-0b5b-4ddd-8f53-5cbcff4f0f1f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.306946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:05.374807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068ea59-1e66-4f65-b48f-6fd03bb863e5 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.416863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fe80a-cd05-4b30-96ac-3828c701e42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.417698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fe80a-cd05-4b30-96ac-3828c701e42b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.417988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fe80a-cd05-4b30-96ac-3828c701e42b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.418269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fe80a-cd05-4b30-96ac-3828c701e42b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.422262 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:05.438266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fe80a-cd05-4b30-96ac-3828c701e42b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-b79fe80a-cd05-4b30-96ac-3828c701e42b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:05.453861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:05.455248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:05.456418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 283/578] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:05.458054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fe80a-cd05-4b30-96ac-3828c701e42b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.468464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df62a58-0b5b-4ddd-8f53-5cbcff4f0f1f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.508776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45494c-d8a8-404e-9613-677d5fbeb611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.509586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45494c-d8a8-404e-9613-677d5fbeb611 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.509864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45494c-d8a8-404e-9613-677d5fbeb611 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.510140 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45494c-d8a8-404e-9613-677d5fbeb611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.521874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45494c-d8a8-404e-9613-677d5fbeb611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.523374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45494c-d8a8-404e-9613-677d5fbeb611 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-4a45494c-d8a8-404e-9613-677d5fbeb611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:05.542942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:05.544804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:05.546288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:05.546288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:05.546288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:05.546288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:05.546288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 296/579] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:05.546288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45494c-d8a8-404e-9613-677d5fbeb611 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.612134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fe80a-cd05-4b30-96ac-3828c701e42b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.651357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefa045-f9f3-430b-aeab-a8dfda25d26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.652079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefa045-f9f3-430b-aeab-a8dfda25d26b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.652362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefa045-f9f3-430b-aeab-a8dfda25d26b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.652668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefa045-f9f3-430b-aeab-a8dfda25d26b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.655137 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:05.661789 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fefa045-f9f3-430b-aeab-a8dfda25d26b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.663285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fefa045-f9f3-430b-aeab-a8dfda25d26b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-8fefa045-f9f3-430b-aeab-a8dfda25d26b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:05.679627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:05.681424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:05.683119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:05.683119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:05.683119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:05.683119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:05.683119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:05.683119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:05.683119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:05.683119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:05.683119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 284/580] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:05.683119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefa045-f9f3-430b-aeab-a8dfda25d26b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.726990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45494c-d8a8-404e-9613-677d5fbeb611 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.758021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:05.771175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28582-c911-479c-8a47-4c68a3bddc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.772101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28582-c911-479c-8a47-4c68a3bddc32 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.772575 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28582-c911-479c-8a47-4c68a3bddc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.772955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28582-c911-479c-8a47-4c68a3bddc32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.785200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca28582-c911-479c-8a47-4c68a3bddc32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.786868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca28582-c911-479c-8a47-4c68a3bddc32 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-8ca28582-c911-479c-8a47-4c68a3bddc32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:05.809529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:05.811356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:05.813142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:05.813142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:05.813142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:05.813142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:05.813142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 297/581] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:05.813715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefa045-f9f3-430b-aeab-a8dfda25d26b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.813854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28582-c911-479c-8a47-4c68a3bddc32 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.851581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b00b8f-011f-47df-95c8-b355474bb15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:05.852253 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b00b8f-011f-47df-95c8-b355474bb15d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:05.852572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b00b8f-011f-47df-95c8-b355474bb15d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:05.852854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b00b8f-011f-47df-95c8-b355474bb15d None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:05.853663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b00b8f-011f-47df-95c8-b355474bb15d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:05.853977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b00b8f-011f-47df-95c8-b355474bb15d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:05.855436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b00b8f-011f-47df-95c8-b355474bb15d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:05.868821 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 285/582] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:05.869688 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b00b8f-011f-47df-95c8-b355474bb15d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:05.967196 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28582-c911-479c-8a47-4c68a3bddc32 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:05.978615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:05.993456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b00b8f-011f-47df-95c8-b355474bb15d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:06.009098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be479e51-56b2-4962-850d-06c12855ed0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:06.009749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be479e51-56b2-4962-850d-06c12855ed0c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:06.010042 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be479e51-56b2-4962-850d-06c12855ed0c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:06.010310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be479e51-56b2-4962-850d-06c12855ed0c None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:06.011069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be479e51-56b2-4962-850d-06c12855ed0c None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:06.011377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be479e51-56b2-4962-850d-06c12855ed0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.013605 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be479e51-56b2-4962-850d-06c12855ed0c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.014584 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:06.023651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 298/583] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:06.024658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be479e51-56b2-4962-850d-06c12855ed0c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.036233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3edb9-885f-4650-93d1-8d426f8fb51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:06.036941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3edb9-885f-4650-93d1-8d426f8fb51d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:06.037228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3edb9-885f-4650-93d1-8d426f8fb51d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:06.037507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3edb9-885f-4650-93d1-8d426f8fb51d None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:06.038246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3edb9-885f-4650-93d1-8d426f8fb51d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:06.038557 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3edb9-885f-4650-93d1-8d426f8fb51d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.039944 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3edb9-885f-4650-93d1-8d426f8fb51d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.054570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 286/584] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:06.055818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3edb9-885f-4650-93d1-8d426f8fb51d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.190800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3edb9-885f-4650-93d1-8d426f8fb51d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:06.191369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be479e51-56b2-4962-850d-06c12855ed0c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:06.235886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2c7e8-e0dd-464a-bf23-00b383aeea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:06.236987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2c7e8-e0dd-464a-bf23-00b383aeea7e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:06.237283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d9cef-1caa-4f29-808f-104da012db8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:06.237497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2c7e8-e0dd-464a-bf23-00b383aeea7e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:06.237993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2c7e8-e0dd-464a-bf23-00b383aeea7e None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:06.238258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d9cef-1caa-4f29-808f-104da012db8f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:06.238549 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d9cef-1caa-4f29-808f-104da012db8f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:06.239044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d9cef-1caa-4f29-808f-104da012db8f None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:06.239134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2c7e8-e0dd-464a-bf23-00b383aeea7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:06.239364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2c7e8-e0dd-464a-bf23-00b383aeea7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.240590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d9cef-1caa-4f29-808f-104da012db8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:06.240776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2c7e8-e0dd-464a-bf23-00b383aeea7e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.241154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d9cef-1caa-4f29-808f-104da012db8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.243911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d9cef-1caa-4f29-808f-104da012db8f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.253532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 287/585] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:06.254378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2c7e8-e0dd-464a-bf23-00b383aeea7e None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.266816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 299/586] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:06.267942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d9cef-1caa-4f29-808f-104da012db8f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.302104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:06.316902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:06.406622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2c7e8-e0dd-464a-bf23-00b383aeea7e None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:06.426282 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d9cef-1caa-4f29-808f-104da012db8f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:06.458426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec195e8-3046-4b02-8f0b-846172fc179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:06.459356 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec195e8-3046-4b02-8f0b-846172fc179a None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:06.459640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec195e8-3046-4b02-8f0b-846172fc179a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:06.459925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec195e8-3046-4b02-8f0b-846172fc179a None admin] PATH_INFO: `/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/c77f6ccc7cb84c04ac532007a04a0f5e/roles/ca5450234bd44b46a10185c4c8a0647f` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:06.472261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f2b79-34f0-4c42-b0c2-451f772b7501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:06.473558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f2b79-34f0-4c42-b0c2-451f772b7501 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:06.474010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f2b79-34f0-4c42-b0c2-451f772b7501 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:06.474385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f2b79-34f0-4c42-b0c2-451f772b7501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:06.480455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec195e8-3046-4b02-8f0b-846172fc179a None admin] RBAC: Authorizing `identity:create_grant(project_id=b0638fe2ce7a40299f2d654f58ed535c, user_id=c77f6ccc7cb84c04ac532007a04a0f5e, role_id=ca5450234bd44b46a10185c4c8a0647f)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.483078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec195e8-3046-4b02-8f0b-846172fc179a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.491952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f2b79-34f0-4c42-b0c2-451f772b7501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.493654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f2b79-34f0-4c42-b0c2-451f772b7501 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.502092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 288/587] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:06 2022] PUT /identity/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/c77f6ccc7cb84c04ac532007a04a0f5e/roles/ca5450234bd44b46a10185c4c8a0647f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:20:06.503253 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec195e8-3046-4b02-8f0b-846172fc179a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b32f2b79-34f0-4c42-b0c2-451f772b7501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:06.515857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:06.527471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:06.529371 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:06.529371 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:06.529371 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:06.529371 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:06.529371 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 300/588] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:06.529371 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f2b79-34f0-4c42-b0c2-451f772b7501 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.668598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:06.677782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:06.699401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec195e8-3046-4b02-8f0b-846172fc179a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:06.713639 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f2b79-34f0-4c42-b0c2-451f772b7501 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:06.765792 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078c4d5-fce8-4955-ac69-558abf4fde16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:06.766968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078c4d5-fce8-4955-ac69-558abf4fde16 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:06.767479 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078c4d5-fce8-4955-ac69-558abf4fde16 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:06.767974 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078c4d5-fce8-4955-ac69-558abf4fde16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:06.777096 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a6d23-60e1-4aff-8af4-691295d69ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:06.779417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a6d23-60e1-4aff-8af4-691295d69ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:06.779920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a6d23-60e1-4aff-8af4-691295d69ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:06.780391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a6d23-60e1-4aff-8af4-691295d69ee4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:06.781318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078c4d5-fce8-4955-ac69-558abf4fde16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.784002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078c4d5-fce8-4955-ac69-558abf4fde16 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.794021 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a6d23-60e1-4aff-8af4-691295d69ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:06.796981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a6d23-60e1-4aff-8af4-691295d69ee4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-e078c4d5-fce8-4955-ac69-558abf4fde16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:06.810056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:06.811312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:06.812251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:06.812251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:06.812251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:06.812251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:06.812251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:06.812251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:06.812251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:06.812251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:06.820195 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 289/589] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:06.820195 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078c4d5-fce8-4955-ac69-558abf4fde16 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-487a6d23-60e1-4aff-8af4-691295d69ee4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:06.829727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:06.830945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:06.837067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:06.837067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:06.837067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:06.837067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:06.837067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:06.837067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:06.837067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:06.837067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:06.837067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:20:06.837067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:06.837067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 301/590] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:06.837067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a6d23-60e1-4aff-8af4-691295d69ee4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:06.995259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078c4d5-fce8-4955-ac69-558abf4fde16 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:07.006755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a6d23-60e1-4aff-8af4-691295d69ee4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:07.054520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934da8b6-b498-41de-958f-b28b2239d214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:07.055481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934da8b6-b498-41de-958f-b28b2239d214 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.055941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934da8b6-b498-41de-958f-b28b2239d214 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.056399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934da8b6-b498-41de-958f-b28b2239d214 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.057547 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934da8b6-b498-41de-958f-b28b2239d214 None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=8fec30ef0b8c41dca643cfefebfe904b) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:07.058053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934da8b6-b498-41de-958f-b28b2239d214 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:07.060806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934da8b6-b498-41de-958f-b28b2239d214 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:07.066453 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31837346-4dd6-436e-9c70-bd0727b2a132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:07.066453 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31837346-4dd6-436e-9c70-bd0727b2a132 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.066453 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31837346-4dd6-436e-9c70-bd0727b2a132 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.066453 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31837346-4dd6-436e-9c70-bd0727b2a132 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.067552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31837346-4dd6-436e-9c70-bd0727b2a132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:07.068078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31837346-4dd6-436e-9c70-bd0727b2a132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:07.070682 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31837346-4dd6-436e-9c70-bd0727b2a132 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:07.073641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 290/591] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:06 2022] GET /identity/v3/role_assignments?user.id=8fec30ef0b8c41dca643cfefebfe904b&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:07.075162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934da8b6-b498-41de-958f-b28b2239d214 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:07.100559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 302/592] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:07.102240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31837346-4dd6-436e-9c70-bd0727b2a132 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:07.110104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:07.113843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:07.307162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934da8b6-b498-41de-958f-b28b2239d214 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:07.325250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31837346-4dd6-436e-9c70-bd0727b2a132 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:07.373329 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397c5ef-5499-4725-9a97-2b9f5bc94fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:07.374641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397c5ef-5499-4725-9a97-2b9f5bc94fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.375155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397c5ef-5499-4725-9a97-2b9f5bc94fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.375659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397c5ef-5499-4725-9a97-2b9f5bc94fc5 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.377264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397c5ef-5499-4725-9a97-2b9f5bc94fc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:07.377946 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397c5ef-5499-4725-9a97-2b9f5bc94fc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:07.381087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397c5ef-5499-4725-9a97-2b9f5bc94fc5 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:07.392789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c53373-a8d8-40f2-83f7-b0a75adda42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:07.393913 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c53373-a8d8-40f2-83f7-b0a75adda42d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.394403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c53373-a8d8-40f2-83f7-b0a75adda42d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.394881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c53373-a8d8-40f2-83f7-b0a75adda42d None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.396770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c53373-a8d8-40f2-83f7-b0a75adda42d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:07.397312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c53373-a8d8-40f2-83f7-b0a75adda42d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:07.399956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c53373-a8d8-40f2-83f7-b0a75adda42d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:07.409380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 291/593] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:07.411070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397c5ef-5499-4725-9a97-2b9f5bc94fc5 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:07.452792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 303/594] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jul 21 22:20:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:07.453767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c53373-a8d8-40f2-83f7-b0a75adda42d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:07.546731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:07.558821 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:07.625843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397c5ef-5499-4725-9a97-2b9f5bc94fc5 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:07.642642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c53373-a8d8-40f2-83f7-b0a75adda42d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:07.696213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3daa37-a6ce-42b0-b0e4-032ea83e1710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:07.698258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3daa37-a6ce-42b0-b0e4-032ea83e1710 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.698957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3daa37-a6ce-42b0-b0e4-032ea83e1710 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.699389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3daa37-a6ce-42b0-b0e4-032ea83e1710 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.706824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f560af8-3ae2-46a3-9f2b-4086f5a536b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:07.707843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f560af8-3ae2-46a3-9f2b-4086f5a536b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:07.708136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f560af8-3ae2-46a3-9f2b-4086f5a536b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:07.708405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f560af8-3ae2-46a3-9f2b-4086f5a536b2 None admin] PATH_INFO: `/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/78b67db080874f3ebb508239c7a8ccf8/roles/ca5450234bd44b46a10185c4c8a0647f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:07.718447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3daa37-a6ce-42b0-b0e4-032ea83e1710 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:07.723408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3daa37-a6ce-42b0-b0e4-032ea83e1710 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:07.727489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f560af8-3ae2-46a3-9f2b-4086f5a536b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0638fe2ce7a40299f2d654f58ed535c, user_id=78b67db080874f3ebb508239c7a8ccf8, role_id=ca5450234bd44b46a10185c4c8a0647f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:07.729485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f560af8-3ae2-46a3-9f2b-4086f5a536b2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:07.752524 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 304/595] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:07 2022] PUT /identity/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/78b67db080874f3ebb508239c7a8ccf8/roles/ca5450234bd44b46a10185c4c8a0647f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:20:07.752524 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f560af8-3ae2-46a3-9f2b-4086f5a536b2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-8a3daa37-a6ce-42b0-b0e4-032ea83e1710 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:07.768338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:07.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:07.773958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:07.773958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:07.773958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:07.773958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:07.773958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:07.773958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:07.773958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:07.773958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:07.773958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:20:07.773958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:07.773958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 292/596] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:07.785840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3daa37-a6ce-42b0-b0e4-032ea83e1710 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:08.008767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3daa37-a6ce-42b0-b0e4-032ea83e1710 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:08.050332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:08.056388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f560af8-3ae2-46a3-9f2b-4086f5a536b2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:08.070517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4175d34-7ccb-4252-af84-e95cf106a9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:08.072264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:08.074536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4175d34-7ccb-4252-af84-e95cf106a9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:08.076063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4175d34-7ccb-4252-af84-e95cf106a9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:08.077442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4175d34-7ccb-4252-af84-e95cf106a9c6 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:08.082261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4175d34-7ccb-4252-af84-e95cf106a9c6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:08.083341 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4175d34-7ccb-4252-af84-e95cf106a9c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:08.088109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4175d34-7ccb-4252-af84-e95cf106a9c6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:08.117889 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 293/597] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Jul 21 22:20:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:08.124389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7bf8c-bacf-48cc-9fce-a910717b15c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:08.125332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7bf8c-bacf-48cc-9fce-a910717b15c3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:08.125621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7bf8c-bacf-48cc-9fce-a910717b15c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:08.125892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7bf8c-bacf-48cc-9fce-a910717b15c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:08.129437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4175d34-7ccb-4252-af84-e95cf106a9c6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:08.161393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7bf8c-bacf-48cc-9fce-a910717b15c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:08.161393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7bf8c-bacf-48cc-9fce-a910717b15c3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b9c7bf8c-bacf-48cc-9fce-a910717b15c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:08.197086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:08.198221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:08.200625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:08.200625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:08.200625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:08.200625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:08.200625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 305/598] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:08.212070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7bf8c-bacf-48cc-9fce-a910717b15c3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:08.335459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4175d34-7ccb-4252-af84-e95cf106a9c6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:08.367749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7bf8c-bacf-48cc-9fce-a910717b15c3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:08.407930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce7890-5f14-4b95-8c58-7d5d4ed83919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:08.408657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce7890-5f14-4b95-8c58-7d5d4ed83919 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:08.408968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce7890-5f14-4b95-8c58-7d5d4ed83919 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:08.409260 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce7890-5f14-4b95-8c58-7d5d4ed83919 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:08.409978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce7890-5f14-4b95-8c58-7d5d4ed83919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:08.410386 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce7890-5f14-4b95-8c58-7d5d4ed83919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:08.411815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce7890-5f14-4b95-8c58-7d5d4ed83919 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:08.421647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e74be-a52f-4ba3-80d7-2e38197a2ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:08.423488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e74be-a52f-4ba3-80d7-2e38197a2ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:08.424268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e74be-a52f-4ba3-80d7-2e38197a2ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:08.425062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e74be-a52f-4ba3-80d7-2e38197a2ec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:08.432419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 306/599] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:08.453382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e74be-a52f-4ba3-80d7-2e38197a2ec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:08.454870 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e74be-a52f-4ba3-80d7-2e38197a2ec5 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:08.457400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce7890-5f14-4b95-8c58-7d5d4ed83919 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-701e74be-a52f-4ba3-80d7-2e38197a2ec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:08.491442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:08.492966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:08.494466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:08.494466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:08.494466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:08.494466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:08.494466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 294/600] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:08.501561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e74be-a52f-4ba3-80d7-2e38197a2ec5 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:08.509429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:08.552430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:08.668565 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e74be-a52f-4ba3-80d7-2e38197a2ec5 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:08.691371 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce7890-5f14-4b95-8c58-7d5d4ed83919 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:08.738912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59cbc4-fe8c-411a-ad78-84cf9b146e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:08.739593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59cbc4-fe8c-411a-ad78-84cf9b146e3a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:08.739989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59cbc4-fe8c-411a-ad78-84cf9b146e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:08.740272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59cbc4-fe8c-411a-ad78-84cf9b146e3a None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:08.741173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59cbc4-fe8c-411a-ad78-84cf9b146e3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:08.741489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59cbc4-fe8c-411a-ad78-84cf9b146e3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:08.742909 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59cbc4-fe8c-411a-ad78-84cf9b146e3a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:08.756966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 295/601] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:08.761295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce779c-bb75-46e9-91bc-05d34ae59f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:08.763616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59cbc4-fe8c-411a-ad78-84cf9b146e3a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:08.764532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce779c-bb75-46e9-91bc-05d34ae59f62 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:08.765005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce779c-bb75-46e9-91bc-05d34ae59f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:08.765424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce779c-bb75-46e9-91bc-05d34ae59f62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:08.792309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce779c-bb75-46e9-91bc-05d34ae59f62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:08.794747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce779c-bb75-46e9-91bc-05d34ae59f62 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-9dce779c-bb75-46e9-91bc-05d34ae59f62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:08.831770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:08.833244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:08.834482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:08.834482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:08.834482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:08.834482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:08.834482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:08.834482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:08.834482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:08.834482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:08.834482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 307/602] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:08.844470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce779c-bb75-46e9-91bc-05d34ae59f62 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:08.971037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:08.975012 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59cbc4-fe8c-411a-ad78-84cf9b146e3a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:09.013660 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c3493-de82-4159-ac56-b180561b6a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:09.014548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c3493-de82-4159-ac56-b180561b6a93 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.014837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c3493-de82-4159-ac56-b180561b6a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.015119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c3493-de82-4159-ac56-b180561b6a93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.023106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c3493-de82-4159-ac56-b180561b6a93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:09.024632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c3493-de82-4159-ac56-b180561b6a93 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-307c3493-de82-4159-ac56-b180561b6a93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:09.039984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:09.040990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:09.042090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:09.042090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:09.042090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:09.042090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:09.042090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:09.042090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:09.042090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:09.042090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:09.042090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 296/603] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:09.054472 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c3493-de82-4159-ac56-b180561b6a93 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:09.074071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:09.096581 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce779c-bb75-46e9-91bc-05d34ae59f62 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:09.136776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85314e2a-7007-4128-860f-41e8ef5ad96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:09.138214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85314e2a-7007-4128-860f-41e8ef5ad96e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.138601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85314e2a-7007-4128-860f-41e8ef5ad96e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.138977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85314e2a-7007-4128-860f-41e8ef5ad96e None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.140302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85314e2a-7007-4128-860f-41e8ef5ad96e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:09.140978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85314e2a-7007-4128-860f-41e8ef5ad96e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:09.143726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85314e2a-7007-4128-860f-41e8ef5ad96e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:09.167470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 308/604] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:09.169661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a927bee-8c00-4cf6-8259-c7170f1ff496 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.170079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a927bee-8c00-4cf6-8259-c7170f1ff496 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.170464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a927bee-8c00-4cf6-8259-c7170f1ff496 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.176438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 309/605] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:09.178054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5399fe2-71bd-4608-bf60-cbf051824e8e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.178457 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5399fe2-71bd-4608-bf60-cbf051824e8e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.178839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5399fe2-71bd-4608-bf60-cbf051824e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.198186 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b5399fe2-71bd-4608-bf60-cbf051824e8e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:09.199617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b5399fe2-71bd-4608-bf60-cbf051824e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:09.284994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c3493-de82-4159-ac56-b180561b6a93 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:09.288968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 310/606] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:09.297496 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5399fe2-71bd-4608-bf60-cbf051824e8e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:09.363703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1bada-4b2b-42a0-a46e-135bf1cd0872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:09.364621 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1bada-4b2b-42a0-a46e-135bf1cd0872 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.365011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1bada-4b2b-42a0-a46e-135bf1cd0872 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.365391 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1bada-4b2b-42a0-a46e-135bf1cd0872 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.366324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1bada-4b2b-42a0-a46e-135bf1cd0872 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:09.367448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1bada-4b2b-42a0-a46e-135bf1cd0872 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:09.369176 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1bada-4b2b-42a0-a46e-135bf1cd0872 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:09.386451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 297/607] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:09.388627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1bada-4b2b-42a0-a46e-135bf1cd0872 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:09.418677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:09.441684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:09.470836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5399fe2-71bd-4608-bf60-cbf051824e8e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:09.509748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d5587-c380-4e14-95d8-c107c9f04376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:09.510767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d5587-c380-4e14-95d8-c107c9f04376 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.511116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d5587-c380-4e14-95d8-c107c9f04376 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.511458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d5587-c380-4e14-95d8-c107c9f04376 None admin] PATH_INFO: `/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/a697d1e7e42048a8a1fb997bcc7b3fab/roles/ca5450234bd44b46a10185c4c8a0647f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.533091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d5587-c380-4e14-95d8-c107c9f04376 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0638fe2ce7a40299f2d654f58ed535c, user_id=a697d1e7e42048a8a1fb997bcc7b3fab, role_id=ca5450234bd44b46a10185c4c8a0647f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:09.534760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d5587-c380-4e14-95d8-c107c9f04376 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:09.553615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 311/608] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:09 2022] PUT /identity/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/a697d1e7e42048a8a1fb997bcc7b3fab/roles/ca5450234bd44b46a10185c4c8a0647f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:20:09.554647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d5587-c380-4e14-95d8-c107c9f04376 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:09.647807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1bada-4b2b-42a0-a46e-135bf1cd0872 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:09.677273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d5587-c380-4e14-95d8-c107c9f04376 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:09.705130 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:09.721010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804044d-199c-41cb-a4f9-0ad204da066d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:09.722033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804044d-199c-41cb-a4f9-0ad204da066d None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.722450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804044d-199c-41cb-a4f9-0ad204da066d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.722752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804044d-199c-41cb-a4f9-0ad204da066d None admin] PATH_INFO: `/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/8a120d9e7e204dc5b972fd9402897b41/roles/ca5450234bd44b46a10185c4c8a0647f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.741892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804044d-199c-41cb-a4f9-0ad204da066d None admin] RBAC: Authorizing `identity:create_grant(project_id=b0638fe2ce7a40299f2d654f58ed535c, user_id=8a120d9e7e204dc5b972fd9402897b41, role_id=ca5450234bd44b46a10185c4c8a0647f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:09.742781 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13a4cc-4e7b-41a3-83d2-e078fbc8680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:09.743825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13a4cc-4e7b-41a3-83d2-e078fbc8680a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.744286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13a4cc-4e7b-41a3-83d2-e078fbc8680a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.744761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13a4cc-4e7b-41a3-83d2-e078fbc8680a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.744922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804044d-199c-41cb-a4f9-0ad204da066d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:09.762937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13a4cc-4e7b-41a3-83d2-e078fbc8680a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:09.768548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 312/609] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:09 2022] PUT /identity/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/8a120d9e7e204dc5b972fd9402897b41/roles/ca5450234bd44b46a10185c4c8a0647f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:20:09.768548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47657e7d-2043-4cd0-a81b-c03333b1efe4 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.768548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47657e7d-2043-4cd0-a81b-c03333b1efe4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.768548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47657e7d-2043-4cd0-a81b-c03333b1efe4 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.768548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 313/610] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-3f13a4cc-4e7b-41a3-83d2-e078fbc8680a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:09.783043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:09.784338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:09.785544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:09.785544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:09.785544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:09.785544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:09.785544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:09.785544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:09.785544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:09.785544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:09.785544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:09.785544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:09.785544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:09.785544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:09.788724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 298/611] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:09.790458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a49810-93ff-49e1-b60f-c54598ea6efa None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:09.790917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a49810-93ff-49e1-b60f-c54598ea6efa None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:09.791308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a49810-93ff-49e1-b60f-c54598ea6efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:09.799312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13a4cc-4e7b-41a3-83d2-e078fbc8680a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:09.844698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a0a49810-93ff-49e1-b60f-c54598ea6efa None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:09.846566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a0a49810-93ff-49e1-b60f-c54598ea6efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:09.971485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 314/612] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:09.989029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:09.998413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a49810-93ff-49e1-b60f-c54598ea6efa None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:10.030661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13a4cc-4e7b-41a3-83d2-e078fbc8680a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:10.111888 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64727db-480d-4ff7-978f-a2d694abdb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:10.112970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64727db-480d-4ff7-978f-a2d694abdb84 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:10.113771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64727db-480d-4ff7-978f-a2d694abdb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:10.114015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64727db-480d-4ff7-978f-a2d694abdb84 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:10.115572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64727db-480d-4ff7-978f-a2d694abdb84 None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:10.116089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64727db-480d-4ff7-978f-a2d694abdb84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:10.119278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64727db-480d-4ff7-978f-a2d694abdb84 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:10.138415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 299/613] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:10.144142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64727db-480d-4ff7-978f-a2d694abdb84 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:10.272240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a49810-93ff-49e1-b60f-c54598ea6efa None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:10.282075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:10.366824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64727db-480d-4ff7-978f-a2d694abdb84 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:10.401713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0940f-c8a1-4e0f-8518-1ea6bdcce9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:10.412512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0940f-c8a1-4e0f-8518-1ea6bdcce9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:10.413734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0940f-c8a1-4e0f-8518-1ea6bdcce9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:10.414501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0940f-c8a1-4e0f-8518-1ea6bdcce9c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:10.450605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c428bfa-e41e-443d-907d-826bef6902f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:10.453893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428bfa-e41e-443d-907d-826bef6902f2 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:10.453893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428bfa-e41e-443d-907d-826bef6902f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:10.453893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428bfa-e41e-443d-907d-826bef6902f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:10.455536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0940f-c8a1-4e0f-8518-1ea6bdcce9c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-30f0940f-c8a1-4e0f-8518-1ea6bdcce9c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:10.474834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:10.476684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:10.478314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 315/614] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 481 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:10.485967 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c428bfa-e41e-443d-907d-826bef6902f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:10.488822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c428bfa-e41e-443d-907d-826bef6902f2 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:10.490958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0940f-c8a1-4e0f-8518-1ea6bdcce9c9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:10.496932 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-6c428bfa-e41e-443d-907d-826bef6902f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:10.553954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:10.558588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:10.564609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:10.564609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:10.564609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:10.564609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:10.564609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 300/615] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:10.564609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c428bfa-e41e-443d-907d-826bef6902f2 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:10.722903 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0940f-c8a1-4e0f-8518-1ea6bdcce9c9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:10.784044 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c428bfa-e41e-443d-907d-826bef6902f2 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:10.804068 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ea1de-ac70-439b-b860-34671deed93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:10.805176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ea1de-ac70-439b-b860-34671deed93a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:10.805708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ea1de-ac70-439b-b860-34671deed93a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:10.806165 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ea1de-ac70-439b-b860-34671deed93a None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:10.807134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ea1de-ac70-439b-b860-34671deed93a None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:10.807626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ea1de-ac70-439b-b860-34671deed93a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:10.813409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ea1de-ac70-439b-b860-34671deed93a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:10.830179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 316/616] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:10.838567 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ea1de-ac70-439b-b860-34671deed93a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:10.877854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd9e49-43f9-40fd-ac20-3365a0733988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:10.878566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd9e49-43f9-40fd-ac20-3365a0733988 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:10.878859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd9e49-43f9-40fd-ac20-3365a0733988 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:10.879139 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd9e49-43f9-40fd-ac20-3365a0733988 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:10.879890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd9e49-43f9-40fd-ac20-3365a0733988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:10.880199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd9e49-43f9-40fd-ac20-3365a0733988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:10.881640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd9e49-43f9-40fd-ac20-3365a0733988 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:10.892744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:10.905606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 301/617] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:10.922530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd9e49-43f9-40fd-ac20-3365a0733988 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:11.066906 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:11.072959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ea1de-ac70-439b-b860-34671deed93a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:11.146161 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df26b9-7ce7-433c-8d1a-ae0a071e61b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:11.148263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df26b9-7ce7-433c-8d1a-ae0a071e61b3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:11.149133 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df26b9-7ce7-433c-8d1a-ae0a071e61b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:11.149960 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df26b9-7ce7-433c-8d1a-ae0a071e61b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:11.173122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df26b9-7ce7-433c-8d1a-ae0a071e61b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:11.176436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df26b9-7ce7-433c-8d1a-ae0a071e61b3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:11.192307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd9e49-43f9-40fd-ac20-3365a0733988 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-92df26b9-7ce7-433c-8d1a-ae0a071e61b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:11.215617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:11.217444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:11.220005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:11.220005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:11.220005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:11.220005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:11.223700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 317/618] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:11.234223 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df26b9-7ce7-433c-8d1a-ae0a071e61b3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:11.321772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b8b9e-5dde-44c4-90d9-81a658fd466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:11.323071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b8b9e-5dde-44c4-90d9-81a658fd466d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:11.323592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b8b9e-5dde-44c4-90d9-81a658fd466d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:11.324344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b8b9e-5dde-44c4-90d9-81a658fd466d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:11.354188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b8b9e-5dde-44c4-90d9-81a658fd466d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:11.357510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b8b9e-5dde-44c4-90d9-81a658fd466d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:11.404871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-6c4b8b9e-5dde-44c4-90d9-81a658fd466d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:11.410136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:11.411230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:11.412199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:11.412199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:11.412199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:11.412199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:11.412199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:11.412199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:11.412199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:11.412199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:11.412199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:11.412199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:11.421771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 302/619] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 22:20:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 500 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:11.423803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b8b9e-5dde-44c4-90d9-81a658fd466d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:11.449375 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df26b9-7ce7-433c-8d1a-ae0a071e61b3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:11.517712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773b2d6-2244-421c-be33-122b178aeee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:11.518971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b2d6-2244-421c-be33-122b178aeee2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:11.519531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b2d6-2244-421c-be33-122b178aeee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:11.520071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b2d6-2244-421c-be33-122b178aeee2 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:11.522044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773b2d6-2244-421c-be33-122b178aeee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:11.522667 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773b2d6-2244-421c-be33-122b178aeee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:11.525537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773b2d6-2244-421c-be33-122b178aeee2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:11.550356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 318/620] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:11.559415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327084-0873-4dfb-97aa-e316471875a6 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:11.559920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327084-0873-4dfb-97aa-e316471875a6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:11.560407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327084-0873-4dfb-97aa-e316471875a6 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:11.563196 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 319/621] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:11 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:11.567379 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327084-0873-4dfb-97aa-e316471875a6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:11.619942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b8b9e-5dde-44c4-90d9-81a658fd466d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:11.651517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:11.689597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ccc8f-2419-4800-9a14-91b08fa1bfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:11.691185 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ccc8f-2419-4800-9a14-91b08fa1bfa4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:11.691880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ccc8f-2419-4800-9a14-91b08fa1bfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:11.692613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ccc8f-2419-4800-9a14-91b08fa1bfa4 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:11.694443 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ccc8f-2419-4800-9a14-91b08fa1bfa4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:11.695250 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ccc8f-2419-4800-9a14-91b08fa1bfa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:11.697705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ccc8f-2419-4800-9a14-91b08fa1bfa4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:11.734237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 303/622] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 22:20:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:11.736908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d8af9-a380-416e-989e-f80841e9dc7c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:11.737628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d8af9-a380-416e-989e-f80841e9dc7c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:11.738330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d8af9-a380-416e-989e-f80841e9dc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:11.768996 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-445d8af9-a380-416e-989e-f80841e9dc7c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:11.771637 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-445d8af9-a380-416e-989e-f80841e9dc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:11.772448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327084-0873-4dfb-97aa-e316471875a6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:11.838942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d43ed-3d7b-462c-bd9c-40d747ece521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:11.841048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d43ed-3d7b-462c-bd9c-40d747ece521 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:11.842610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d43ed-3d7b-462c-bd9c-40d747ece521 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:11.842610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d43ed-3d7b-462c-bd9c-40d747ece521 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:11.857675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d43ed-3d7b-462c-bd9c-40d747ece521 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:11.861027 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d43ed-3d7b-462c-bd9c-40d747ece521 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:11.864877 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-da7d43ed-3d7b-462c-bd9c-40d747ece521 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:11.904514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:11.906615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:11.908786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:11.908786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:11.908786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:11.908786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:11.908786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:11.908786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:11.908786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:11.908786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:11.908786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:11.908786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:11.911933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 304/624] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:11.913906 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d8af9-a380-416e-989e-f80841e9dc7c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:11.916801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 320/624] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:11.921172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d43ed-3d7b-462c-bd9c-40d747ece521 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.126889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d43ed-3d7b-462c-bd9c-40d747ece521 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.154399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d8af9-a380-416e-989e-f80841e9dc7c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.158745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:12.198440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99368e83-4aaf-4a7a-a22d-3e5f2c671a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.199285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99368e83-4aaf-4a7a-a22d-3e5f2c671a22 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.199616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99368e83-4aaf-4a7a-a22d-3e5f2c671a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.199944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99368e83-4aaf-4a7a-a22d-3e5f2c671a22 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.200935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99368e83-4aaf-4a7a-a22d-3e5f2c671a22 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:12.201241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99368e83-4aaf-4a7a-a22d-3e5f2c671a22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:12.202707 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99368e83-4aaf-4a7a-a22d-3e5f2c671a22 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:12.223187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451171f0-0dc0-4585-bbdf-7e2cba0c2623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.224377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 321/625] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jul 21 22:20:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:12.224920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451171f0-0dc0-4585-bbdf-7e2cba0c2623 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.225613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451171f0-0dc0-4585-bbdf-7e2cba0c2623 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.225804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99368e83-4aaf-4a7a-a22d-3e5f2c671a22 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.226662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451171f0-0dc0-4585-bbdf-7e2cba0c2623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.327669 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:12.343274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451171f0-0dc0-4585-bbdf-7e2cba0c2623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:12.345704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451171f0-0dc0-4585-bbdf-7e2cba0c2623 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:12.407640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99368e83-4aaf-4a7a-a22d-3e5f2c671a22 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.477358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780e592-3d64-4894-8bb7-94b9f9e8a5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.478729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780e592-3d64-4894-8bb7-94b9f9e8a5ab None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.479219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780e592-3d64-4894-8bb7-94b9f9e8a5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.479520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780e592-3d64-4894-8bb7-94b9f9e8a5ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.485532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 305/626] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:20:12.486829 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2e676-8809-441a-ab68-ebfa8293d7fc None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.487116 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2e676-8809-441a-ab68-ebfa8293d7fc None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.487394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2e676-8809-441a-ab68-ebfa8293d7fc None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.492631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 306/627] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:12.492631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2e676-8809-441a-ab68-ebfa8293d7fc None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.496912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:12.501304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780e592-3d64-4894-8bb7-94b9f9e8a5ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-8780e592-3d64-4894-8bb7-94b9f9e8a5ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:12.509377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:12.512778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 322/628] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:12.515219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780e592-3d64-4894-8bb7-94b9f9e8a5ab None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.641120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780e592-3d64-4894-8bb7-94b9f9e8a5ab None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.660156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2e676-8809-441a-ab68-ebfa8293d7fc None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.666790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:12.697971 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba780e6-5c50-4751-9650-4ee9d413c17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.698698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba780e6-5c50-4751-9650-4ee9d413c17b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.698980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba780e6-5c50-4751-9650-4ee9d413c17b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.699325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba780e6-5c50-4751-9650-4ee9d413c17b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.705443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef98bc4-51f4-43cb-a921-16274e0c9ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.706805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef98bc4-51f4-43cb-a921-16274e0c9ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.707314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef98bc4-51f4-43cb-a921-16274e0c9ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.707817 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef98bc4-51f4-43cb-a921-16274e0c9ba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.712396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba780e6-5c50-4751-9650-4ee9d413c17b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:12.713892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba780e6-5c50-4751-9650-4ee9d413c17b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:12.728095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef98bc4-51f4-43cb-a921-16274e0c9ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:12.730556 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef98bc4-51f4-43cb-a921-16274e0c9ba1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-cba780e6-5c50-4751-9650-4ee9d413c17b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:12.733626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:12.735297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:12.737143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:12.737143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:12.737143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:12.737143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:12.737143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 307/629] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:12.737143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508b633-3dd5-4bff-8bf0-f2077a8a1bf7 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.737143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508b633-3dd5-4bff-8bf0-f2077a8a1bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.737143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508b633-3dd5-4bff-8bf0-f2077a8a1bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.756408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f508b633-3dd5-4bff-8bf0-f2077a8a1bf7 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:12.757661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f508b633-3dd5-4bff-8bf0-f2077a8a1bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-2ef98bc4-51f4-43cb-a921-16274e0c9ba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:12.773957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:12.775252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:12.777485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:12.777485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:12.777485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:12.777485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:12.777485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 323/630] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:12.777485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef98bc4-51f4-43cb-a921-16274e0c9ba1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.838817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:12.870052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 308/631] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:12.873521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508b633-3dd5-4bff-8bf0-f2077a8a1bf7 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:12.941488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef98bc4-51f4-43cb-a921-16274e0c9ba1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:12.978615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff64326-2ed2-4edc-8490-8a294038fdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:12.979193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff64326-2ed2-4edc-8490-8a294038fdce None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:12.979475 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff64326-2ed2-4edc-8490-8a294038fdce None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:12.979751 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff64326-2ed2-4edc-8490-8a294038fdce None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:12.980555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff64326-2ed2-4edc-8490-8a294038fdce None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:12.980935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff64326-2ed2-4edc-8490-8a294038fdce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:12.982625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff64326-2ed2-4edc-8490-8a294038fdce None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:12.991578 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 324/632] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:12.992359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff64326-2ed2-4edc-8490-8a294038fdce None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.011216 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508b633-3dd5-4bff-8bf0-f2077a8a1bf7 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.026569 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.049320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17dfa7d-e1bd-478b-b756-562e34ca66da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.050017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dfa7d-e1bd-478b-b756-562e34ca66da None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.050300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dfa7d-e1bd-478b-b756-562e34ca66da None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.050580 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17dfa7d-e1bd-478b-b756-562e34ca66da None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.051359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dfa7d-e1bd-478b-b756-562e34ca66da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:13.051671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dfa7d-e1bd-478b-b756-562e34ca66da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.053195 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17dfa7d-e1bd-478b-b756-562e34ca66da None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.067300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 309/633] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:13.068786 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f123d1-efe9-4011-9ecb-96a4fcd5300b None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.069158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f123d1-efe9-4011-9ecb-96a4fcd5300b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.069446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f123d1-efe9-4011-9ecb-96a4fcd5300b None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.070888 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 310/634] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:13.071735 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f123d1-efe9-4011-9ecb-96a4fcd5300b None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.121600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff64326-2ed2-4edc-8490-8a294038fdce None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.133819 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.165682 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee5dac-d292-4b81-b27e-ff80b7e1666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.166472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee5dac-d292-4b81-b27e-ff80b7e1666c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.166843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee5dac-d292-4b81-b27e-ff80b7e1666c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.167126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee5dac-d292-4b81-b27e-ff80b7e1666c None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.167900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee5dac-d292-4b81-b27e-ff80b7e1666c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:13.168838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee5dac-d292-4b81-b27e-ff80b7e1666c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.170292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee5dac-d292-4b81-b27e-ff80b7e1666c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.185283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 325/635] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:13.186337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee5dac-d292-4b81-b27e-ff80b7e1666c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.212953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f123d1-efe9-4011-9ecb-96a4fcd5300b None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.251228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ec841-75c3-4d13-b2a8-05266afe8d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.251954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ec841-75c3-4d13-b2a8-05266afe8d20 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.252238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ec841-75c3-4d13-b2a8-05266afe8d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.252550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ec841-75c3-4d13-b2a8-05266afe8d20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.268595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ec841-75c3-4d13-b2a8-05266afe8d20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-e09ec841-75c3-4d13-b2a8-05266afe8d20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:13.277092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:13.278578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:13.279581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 311/636] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:13.280322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ec841-75c3-4d13-b2a8-05266afe8d20 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.294504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.311210 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee5dac-d292-4b81-b27e-ff80b7e1666c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.354482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9433d2-dc3b-4bf9-8d70-b7ead9b22308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.356143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9433d2-dc3b-4bf9-8d70-b7ead9b22308 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.357344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9433d2-dc3b-4bf9-8d70-b7ead9b22308 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.357707 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9433d2-dc3b-4bf9-8d70-b7ead9b22308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.369826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9433d2-dc3b-4bf9-8d70-b7ead9b22308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.372006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9433d2-dc3b-4bf9-8d70-b7ead9b22308 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ed9433d2-dc3b-4bf9-8d70-b7ead9b22308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:13.393309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:13.394407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:13.400507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:13.400507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:13.400507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:13.400507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:13.400507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 326/637] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:13.400507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d94c4-14ab-4635-b1a0-0f3a1d7219e4 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.400507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d94c4-14ab-4635-b1a0-0f3a1d7219e4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.400507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d94c4-14ab-4635-b1a0-0f3a1d7219e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.415714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-bc2d94c4-14ab-4635-b1a0-0f3a1d7219e4 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:13.416255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bc2d94c4-14ab-4635-b1a0-0f3a1d7219e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:13.426505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ec841-75c3-4d13-b2a8-05266afe8d20 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.433003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.478358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f7071-4396-4093-9fa2-ebe60565f602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.479169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f7071-4396-4093-9fa2-ebe60565f602 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.479454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f7071-4396-4093-9fa2-ebe60565f602 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.479733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f7071-4396-4093-9fa2-ebe60565f602 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.488884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f7071-4396-4093-9fa2-ebe60565f602 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.490501 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f7071-4396-4093-9fa2-ebe60565f602 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-b74f7071-4396-4093-9fa2-ebe60565f602 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:13.507521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:13.509380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:13.517396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:13.517396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:13.517396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:13.517396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:13.517396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:13.517396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:13.517396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:13.517396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:13.517845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 327/638] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:13.517845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d94c4-14ab-4635-b1a0-0f3a1d7219e4 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.518302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 312/639] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:13.518302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f7071-4396-4093-9fa2-ebe60565f602 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.611488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.654281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d94c4-14ab-4635-b1a0-0f3a1d7219e4 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.667520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f7071-4396-4093-9fa2-ebe60565f602 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.701440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b73f0-1ecd-4056-83d5-fa5f07c06baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.702247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b73f0-1ecd-4056-83d5-fa5f07c06baf None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.702527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b73f0-1ecd-4056-83d5-fa5f07c06baf None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.702800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b73f0-1ecd-4056-83d5-fa5f07c06baf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.710213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b73f0-1ecd-4056-83d5-fa5f07c06baf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.710794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45401211-f090-4ff0-a20f-ea50c0b9c766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.711647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45401211-f090-4ff0-a20f-ea50c0b9c766 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.711770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b73f0-1ecd-4056-83d5-fa5f07c06baf None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.711985 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45401211-f090-4ff0-a20f-ea50c0b9c766 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.712358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45401211-f090-4ff0-a20f-ea50c0b9c766 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.713355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45401211-f090-4ff0-a20f-ea50c0b9c766 None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:13.713779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45401211-f090-4ff0-a20f-ea50c0b9c766 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.717330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45401211-f090-4ff0-a20f-ea50c0b9c766 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-d45b73f0-1ecd-4056-83d5-fa5f07c06baf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:13.727302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:13.729108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:13.731367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:13.731367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:13.731367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:13.731367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:13.731367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:13.731367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:13.731367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:13.731367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:13.731367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 328/640] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:13.731367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b73f0-1ecd-4056-83d5-fa5f07c06baf None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.732429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 313/641] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:13.732429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45401211-f090-4ff0-a20f-ea50c0b9c766 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.763987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.860134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45401211-f090-4ff0-a20f-ea50c0b9c766 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.863313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b73f0-1ecd-4056-83d5-fa5f07c06baf None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:13.896997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:13.901134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848995b-d625-44ba-a5b4-ceae19f68ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.901822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848995b-d625-44ba-a5b4-ceae19f68ade None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.902118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848995b-d625-44ba-a5b4-ceae19f68ade None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.902971 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848995b-d625-44ba-a5b4-ceae19f68ade None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.904015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848995b-d625-44ba-a5b4-ceae19f68ade None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:13.904340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848995b-d625-44ba-a5b4-ceae19f68ade None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.905873 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d848995b-d625-44ba-a5b4-ceae19f68ade None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.909354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b3dda-ac11-488a-8cd6-fbf2ef389124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:13.909354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b3dda-ac11-488a-8cd6-fbf2ef389124 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:13.909354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b3dda-ac11-488a-8cd6-fbf2ef389124 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:13.909354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b3dda-ac11-488a-8cd6-fbf2ef389124 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:13.909962 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b3dda-ac11-488a-8cd6-fbf2ef389124 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:13.909962 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b3dda-ac11-488a-8cd6-fbf2ef389124 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:13.911197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b3dda-ac11-488a-8cd6-fbf2ef389124 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:13.921423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 314/642] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:13.922111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848995b-d625-44ba-a5b4-ceae19f68ade None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:13.933462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 329/643] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:13.934398 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b3dda-ac11-488a-8cd6-fbf2ef389124 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.032705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:14.056293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848995b-d625-44ba-a5b4-ceae19f68ade None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.076204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b3dda-ac11-488a-8cd6-fbf2ef389124 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.103661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855088d6-c4c8-4ccc-8404-07525686928d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.104529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855088d6-c4c8-4ccc-8404-07525686928d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.104815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855088d6-c4c8-4ccc-8404-07525686928d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.105097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855088d6-c4c8-4ccc-8404-07525686928d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.117770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855088d6-c4c8-4ccc-8404-07525686928d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-855088d6-c4c8-4ccc-8404-07525686928d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:14.125953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:14.127827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 315/644] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:14.129576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855088d6-c4c8-4ccc-8404-07525686928d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.134988 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888b5a6-82ed-4d5b-8319-75332d224692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.135941 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888b5a6-82ed-4d5b-8319-75332d224692 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.136313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888b5a6-82ed-4d5b-8319-75332d224692 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.137275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888b5a6-82ed-4d5b-8319-75332d224692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.149248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888b5a6-82ed-4d5b-8319-75332d224692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.150750 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888b5a6-82ed-4d5b-8319-75332d224692 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-3888b5a6-82ed-4d5b-8319-75332d224692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:14.170365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:14.172510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:14.174387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:14.174387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:14.174387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:14.174387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:14.174387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 330/645] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:14.174387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888b5a6-82ed-4d5b-8319-75332d224692 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.229843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:14.282507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855088d6-c4c8-4ccc-8404-07525686928d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.307138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888b5a6-82ed-4d5b-8319-75332d224692 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.321495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712665e-9b86-4984-bbfd-158afe808e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.322198 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712665e-9b86-4984-bbfd-158afe808e18 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.322485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712665e-9b86-4984-bbfd-158afe808e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.322791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712665e-9b86-4984-bbfd-158afe808e18 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.323607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712665e-9b86-4984-bbfd-158afe808e18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:14.323919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712665e-9b86-4984-bbfd-158afe808e18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.325551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712665e-9b86-4984-bbfd-158afe808e18 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.338509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 316/646] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:14.339380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712665e-9b86-4984-bbfd-158afe808e18 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.349543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:14.375592 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f7982-0fb4-4cd3-8f36-6912a3af8d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.376539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f7982-0fb4-4cd3-8f36-6912a3af8d0e None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.376812 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f7982-0fb4-4cd3-8f36-6912a3af8d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.377113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f7982-0fb4-4cd3-8f36-6912a3af8d0e None admin] PATH_INFO: `/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/8fec30ef0b8c41dca643cfefebfe904b/roles/ca5450234bd44b46a10185c4c8a0647f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.397834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f7982-0fb4-4cd3-8f36-6912a3af8d0e None admin] RBAC: Authorizing `identity:create_grant(project_id=b0638fe2ce7a40299f2d654f58ed535c, user_id=8fec30ef0b8c41dca643cfefebfe904b, role_id=ca5450234bd44b46a10185c4c8a0647f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.400254 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f7982-0fb4-4cd3-8f36-6912a3af8d0e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.420913 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 331/647] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:14 2022] PUT /identity/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/8fec30ef0b8c41dca643cfefebfe904b/roles/ca5450234bd44b46a10185c4c8a0647f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:20:14.421819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f7982-0fb4-4cd3-8f36-6912a3af8d0e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.521232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712665e-9b86-4984-bbfd-158afe808e18 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.551811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:14.570333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f7982-0fb4-4cd3-8f36-6912a3af8d0e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.603866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f0171-6a4d-4364-aa03-6166ddc37fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.605353 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f0171-6a4d-4364-aa03-6166ddc37fef None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.606512 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f0171-6a4d-4364-aa03-6166ddc37fef None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.607024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f0171-6a4d-4364-aa03-6166ddc37fef None admin] PATH_INFO: `/v3/users/glance` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.619069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f0171-6a4d-4364-aa03-6166ddc37fef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.621822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f0171-6a4d-4364-aa03-6166ddc37fef None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.624309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cd89d-6880-46c0-b171-2fc7c04235e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.625030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cd89d-6880-46c0-b171-2fc7c04235e8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.625319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cd89d-6880-46c0-b171-2fc7c04235e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.625597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cd89d-6880-46c0-b171-2fc7c04235e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.626361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4cd89d-6880-46c0-b171-2fc7c04235e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:14.626671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4cd89d-6880-46c0-b171-2fc7c04235e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.628105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4cd89d-6880-46c0-b171-2fc7c04235e8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.638056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 332/648] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:14.638794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cd89d-6880-46c0-b171-2fc7c04235e8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-207f0171-6a4d-4364-aa03-6166ddc37fef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:14.644091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:14.645130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:14.646352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:14.646352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:14.646352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:14.646352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:14.646352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:14.646352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:14.646352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:14.646352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:14.646352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:20:14.646352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:14.648271 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 317/649] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:14.649929 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f0171-6a4d-4364-aa03-6166ddc37fef None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.757546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:14.784405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cd89d-6880-46c0-b171-2fc7c04235e8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.802423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f0171-6a4d-4364-aa03-6166ddc37fef None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:14.832035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303d414-5c03-4675-948e-821b698a7160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.832679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303d414-5c03-4675-948e-821b698a7160 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.832971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303d414-5c03-4675-948e-821b698a7160 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.833258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303d414-5c03-4675-948e-821b698a7160 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.834009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303d414-5c03-4675-948e-821b698a7160 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:14.834327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303d414-5c03-4675-948e-821b698a7160 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.835747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303d414-5c03-4675-948e-821b698a7160 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.847074 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2323b8-5ea2-4008-8661-1c6bca0e421f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:14.848178 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2323b8-5ea2-4008-8661-1c6bca0e421f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:14.848667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2323b8-5ea2-4008-8661-1c6bca0e421f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:14.849090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2323b8-5ea2-4008-8661-1c6bca0e421f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:14.849769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 333/650] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:14.850281 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2323b8-5ea2-4008-8661-1c6bca0e421f None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=c77f6ccc7cb84c04ac532007a04a0f5e) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:14.850629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303d414-5c03-4675-948e-821b698a7160 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.850905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2323b8-5ea2-4008-8661-1c6bca0e421f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:14.852543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2323b8-5ea2-4008-8661-1c6bca0e421f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:14.866236 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 318/651] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/role_assignments?user.id=c77f6ccc7cb84c04ac532007a04a0f5e&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:14.868522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2323b8-5ea2-4008-8661-1c6bca0e421f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:14.878151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:15.036110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303d414-5c03-4675-948e-821b698a7160 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:15.045452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2323b8-5ea2-4008-8661-1c6bca0e421f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:15.096394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:15.098539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396a7c3-5528-4fc2-900e-b0612e6f4a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:15.100074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396a7c3-5528-4fc2-900e-b0612e6f4a9f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:15.100656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396a7c3-5528-4fc2-900e-b0612e6f4a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:15.101125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396a7c3-5528-4fc2-900e-b0612e6f4a9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:15.106721 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecafa86-6b37-4ea8-8a92-73e4ee66ed75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:15.107512 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecafa86-6b37-4ea8-8a92-73e4ee66ed75 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:15.107823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecafa86-6b37-4ea8-8a92-73e4ee66ed75 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:15.108088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecafa86-6b37-4ea8-8a92-73e4ee66ed75 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:15.109219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecafa86-6b37-4ea8-8a92-73e4ee66ed75 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:15.109544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecafa86-6b37-4ea8-8a92-73e4ee66ed75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:15.111093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecafa86-6b37-4ea8-8a92-73e4ee66ed75 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:15.125525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396a7c3-5528-4fc2-900e-b0612e6f4a9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:15.127486 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396a7c3-5528-4fc2-900e-b0612e6f4a9f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:15.138479 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 319/652] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:15.139669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecafa86-6b37-4ea8-8a92-73e4ee66ed75 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-1396a7c3-5528-4fc2-900e-b0612e6f4a9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:15.163767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:15.164814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:15.165810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:15.165810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:15.165810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:15.165810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:15.166056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 334/653] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:15.171758 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396a7c3-5528-4fc2-900e-b0612e6f4a9f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:15.302772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecafa86-6b37-4ea8-8a92-73e4ee66ed75 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:15.304890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:15.348598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664f442-0e18-4d70-aa36-f3530540870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:15.349912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664f442-0e18-4d70-aa36-f3530540870c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:15.350417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664f442-0e18-4d70-aa36-f3530540870c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:15.350902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664f442-0e18-4d70-aa36-f3530540870c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:15.355494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396a7c3-5528-4fc2-900e-b0612e6f4a9f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:15.361351 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664f442-0e18-4d70-aa36-f3530540870c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:15.363303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664f442-0e18-4d70-aa36-f3530540870c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-2664f442-0e18-4d70-aa36-f3530540870c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:15.380876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:15.381893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:15.382846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:15.382846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:15.382846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:15.382846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:15.382846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:15.382846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:15.382846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:15.382846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:15.382846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:20:15.382846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:15.391431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 320/654] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:15.392805 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664f442-0e18-4d70-aa36-f3530540870c None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:15.425749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d112bb-d225-4ca1-adf2-9717d8ce44b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:15.426877 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d112bb-d225-4ca1-adf2-9717d8ce44b3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:15.427363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d112bb-d225-4ca1-adf2-9717d8ce44b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:15.428066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d112bb-d225-4ca1-adf2-9717d8ce44b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:15.457219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:15.517527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664f442-0e18-4d70-aa36-f3530540870c None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:15.562456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350c74e-6749-467f-a143-e11da6f00425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:15.563851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350c74e-6749-467f-a143-e11da6f00425 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:15.564388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350c74e-6749-467f-a143-e11da6f00425 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:15.564614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d112bb-d225-4ca1-adf2-9717d8ce44b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:15.565475 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350c74e-6749-467f-a143-e11da6f00425 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:15.566446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350c74e-6749-467f-a143-e11da6f00425 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:15.566978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350c74e-6749-467f-a143-e11da6f00425 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:15.567281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d112bb-d225-4ca1-adf2-9717d8ce44b3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:15.568824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350c74e-6749-467f-a143-e11da6f00425 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:15.588912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 321/655] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:15.592664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350c74e-6749-467f-a143-e11da6f00425 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:15.664863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:15.714612 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 335/656] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:20:15.717809 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d112bb-d225-4ca1-adf2-9717d8ce44b3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:15.786210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350c74e-6749-467f-a143-e11da6f00425 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:15.805958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:15.853226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d112bb-d225-4ca1-adf2-9717d8ce44b3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:15.858259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd30b7-7405-47c8-a23b-5241c8d06ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:15.859398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd30b7-7405-47c8-a23b-5241c8d06ebb None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:15.859895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd30b7-7405-47c8-a23b-5241c8d06ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:15.860396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd30b7-7405-47c8-a23b-5241c8d06ebb None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:15.862101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd30b7-7405-47c8-a23b-5241c8d06ebb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:15.862661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd30b7-7405-47c8-a23b-5241c8d06ebb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:15.865531 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd30b7-7405-47c8-a23b-5241c8d06ebb None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:15.899538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08534690-71ad-4a75-8dc8-c54f60f7c0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:15.900834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08534690-71ad-4a75-8dc8-c54f60f7c0be None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:15.901370 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08534690-71ad-4a75-8dc8-c54f60f7c0be None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:15.901846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08534690-71ad-4a75-8dc8-c54f60f7c0be None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:15.908874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 322/657] 10.222.0.42 () {56 vars in 1137 bytes} [Thu Jul 21 22:20:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:15.910385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd30b7-7405-47c8-a23b-5241c8d06ebb None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:15.912674 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08534690-71ad-4a75-8dc8-c54f60f7c0be None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:15.914384 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08534690-71ad-4a75-8dc8-c54f60f7c0be None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-08534690-71ad-4a75-8dc8-c54f60f7c0be None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:15.935233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:15.936600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:15.937571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:15.937571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:15.937571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:15.937571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:15.937571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:15.937571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:15.937571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:15.937571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:15.937571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:20:15.937571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:15.938147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 336/658] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:15.939162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08534690-71ad-4a75-8dc8-c54f60f7c0be None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:15.986879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:16.087410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd30b7-7405-47c8-a23b-5241c8d06ebb None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:16.107381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08534690-71ad-4a75-8dc8-c54f60f7c0be None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:16.135014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3140d-fe15-4549-b7f0-0395b10ca323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:16.136338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3140d-fe15-4549-b7f0-0395b10ca323 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.137000 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3140d-fe15-4549-b7f0-0395b10ca323 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.137459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3140d-fe15-4549-b7f0-0395b10ca323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.156781 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3140d-fe15-4549-b7f0-0395b10ca323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:16.159572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3140d-fe15-4549-b7f0-0395b10ca323 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:16.170244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8469607-d3e5-4bad-9cbd-4189ffe02f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:16.171309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8469607-d3e5-4bad-9cbd-4189ffe02f12 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.171789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8469607-d3e5-4bad-9cbd-4189ffe02f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.172250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8469607-d3e5-4bad-9cbd-4189ffe02f12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-1ee3140d-fe15-4549-b7f0-0395b10ca323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:16.194519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:16.195454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:16.196337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:16.196337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:16.196337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:16.196337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:16.200511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 323/659] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:16.200511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3140d-fe15-4549-b7f0-0395b10ca323 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:16.234542 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:16.316590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8469607-d3e5-4bad-9cbd-4189ffe02f12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:16.319299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8469607-d3e5-4bad-9cbd-4189ffe02f12 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:16.369175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:16.401829 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3140d-fe15-4549-b7f0-0395b10ca323 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:16.449572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca659fc3-6ed5-4521-b7a0-d2bbd1377ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:16.451121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca659fc3-6ed5-4521-b7a0-d2bbd1377ddf None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.451910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca659fc3-6ed5-4521-b7a0-d2bbd1377ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.452783 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca659fc3-6ed5-4521-b7a0-d2bbd1377ddf None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.454922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca659fc3-6ed5-4521-b7a0-d2bbd1377ddf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:16.455805 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca659fc3-6ed5-4521-b7a0-d2bbd1377ddf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:16.458613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca659fc3-6ed5-4521-b7a0-d2bbd1377ddf None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:16.486172 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 324/660] 10.222.0.42 () {56 vars in 1139 bytes} [Thu Jul 21 22:20:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:16.487484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca659fc3-6ed5-4521-b7a0-d2bbd1377ddf None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:16.514956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 337/661] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:20:16.521994 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8469607-d3e5-4bad-9cbd-4189ffe02f12 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:16.588751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:16.617067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca659fc3-6ed5-4521-b7a0-d2bbd1377ddf None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:16.662120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791b7f2-85da-4392-a1f6-017de3f3e6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:16.663266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791b7f2-85da-4392-a1f6-017de3f3e6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.663789 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791b7f2-85da-4392-a1f6-017de3f3e6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.664365 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791b7f2-85da-4392-a1f6-017de3f3e6c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.665425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791b7f2-85da-4392-a1f6-017de3f3e6c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:16.666070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791b7f2-85da-4392-a1f6-017de3f3e6c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:16.667701 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791b7f2-85da-4392-a1f6-017de3f3e6c3 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:16.682142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 325/662] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:16.684107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791b7f2-85da-4392-a1f6-017de3f3e6c3 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:16.732386 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8469607-d3e5-4bad-9cbd-4189ffe02f12 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:16.791400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:16.818086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dc7e8-7d4f-4c69-84bf-3d000a2e1bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:16.818596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791b7f2-85da-4392-a1f6-017de3f3e6c3 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:16.819246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dc7e8-7d4f-4c69-84bf-3d000a2e1bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.819757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dc7e8-7d4f-4c69-84bf-3d000a2e1bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.820264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dc7e8-7d4f-4c69-84bf-3d000a2e1bc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.860143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755834c9-baaa-47c1-87e2-5b5a6be35a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:16.862038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755834c9-baaa-47c1-87e2-5b5a6be35a89 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:16.862561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755834c9-baaa-47c1-87e2-5b5a6be35a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:16.863062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755834c9-baaa-47c1-87e2-5b5a6be35a89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:16.866823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:16.880058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755834c9-baaa-47c1-87e2-5b5a6be35a89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:16.882318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755834c9-baaa-47c1-87e2-5b5a6be35a89 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-755834c9-baaa-47c1-87e2-5b5a6be35a89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:16.906435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:16.907418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:16.908359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:16.908359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:16.908359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:16.908359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:16.916059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 326/663] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:16.916971 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755834c9-baaa-47c1-87e2-5b5a6be35a89 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:16.973725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dc7e8-7d4f-4c69-84bf-3d000a2e1bc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:16.976893 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dc7e8-7d4f-4c69-84bf-3d000a2e1bc6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:17.089156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755834c9-baaa-47c1-87e2-5b5a6be35a89 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:17.117923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 338/664] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:20:17.118697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dc7e8-7d4f-4c69-84bf-3d000a2e1bc6 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:17.156556 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:17.162967 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85916b-2a31-4b0b-9443-ab3c82a8f58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:17.164510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85916b-2a31-4b0b-9443-ab3c82a8f58a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:17.165026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85916b-2a31-4b0b-9443-ab3c82a8f58a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:17.165588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85916b-2a31-4b0b-9443-ab3c82a8f58a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:17.180051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85916b-2a31-4b0b-9443-ab3c82a8f58a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:17.183114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85916b-2a31-4b0b-9443-ab3c82a8f58a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-af85916b-2a31-4b0b-9443-ab3c82a8f58a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:17.212450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:17.213457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:17.214376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:17.214376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:17.214376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:17.214376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:17.214376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:17.214376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:17.214376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:17.214376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:17.217316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 327/665] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:17.217680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85916b-2a31-4b0b-9443-ab3c82a8f58a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:17.250139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dc7e8-7d4f-4c69-84bf-3d000a2e1bc6 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:17.289852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26da82b-a1dc-4c86-9284-57989f521dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:17.290531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26da82b-a1dc-4c86-9284-57989f521dac None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:17.290814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26da82b-a1dc-4c86-9284-57989f521dac None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:17.291090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26da82b-a1dc-4c86-9284-57989f521dac None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:17.291824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26da82b-a1dc-4c86-9284-57989f521dac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:17.292132 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26da82b-a1dc-4c86-9284-57989f521dac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:17.294272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26da82b-a1dc-4c86-9284-57989f521dac None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:17.311506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 339/666] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:17.311506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c2a33-9794-49ee-8ff8-b1a955e2abfa None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:17.311506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c2a33-9794-49ee-8ff8-b1a955e2abfa None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:17.311506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c2a33-9794-49ee-8ff8-b1a955e2abfa None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:17.311506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 340/667] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:17.312513 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c2a33-9794-49ee-8ff8-b1a955e2abfa None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:17.321561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:17.415280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85916b-2a31-4b0b-9443-ab3c82a8f58a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:17.422400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:17.444000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c2a33-9794-49ee-8ff8-b1a955e2abfa None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:17.483333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148a982-7fa4-4c69-861e-45dd10c42b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:17.484688 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a7156-8fda-4304-b6f8-a6da3a8b8f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:17.484848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148a982-7fa4-4c69-861e-45dd10c42b67 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:17.485318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148a982-7fa4-4c69-861e-45dd10c42b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:17.485424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a7156-8fda-4304-b6f8-a6da3a8b8f28 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:17.485652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a7156-8fda-4304-b6f8-a6da3a8b8f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:17.485824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148a982-7fa4-4c69-861e-45dd10c42b67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:17.485918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a7156-8fda-4304-b6f8-a6da3a8b8f28 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:17.486734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a7156-8fda-4304-b6f8-a6da3a8b8f28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:17.487260 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a7156-8fda-4304-b6f8-a6da3a8b8f28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:17.488924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a7156-8fda-4304-b6f8-a6da3a8b8f28 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:17.508850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 341/668] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:17.510670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a7156-8fda-4304-b6f8-a6da3a8b8f28 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:17.511505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148a982-7fa4-4c69-861e-45dd10c42b67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:17.514675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148a982-7fa4-4c69-861e-45dd10c42b67 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-0148a982-7fa4-4c69-861e-45dd10c42b67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:17.548300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:17.549485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:17.550463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:17.550463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:17.550463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:17.550463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:17.550463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 328/669] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:17.552382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e02d63-e77f-4e38-bd60-e973f19ec88f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:17.552706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e02d63-e77f-4e38-bd60-e973f19ec88f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:17.552987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e02d63-e77f-4e38-bd60-e973f19ec88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:17.576408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-27e02d63-e77f-4e38-bd60-e973f19ec88f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:17.577674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-27e02d63-e77f-4e38-bd60-e973f19ec88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:17.645694 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a7156-8fda-4304-b6f8-a6da3a8b8f28 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:17.668614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 329/670] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:17.668614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e02d63-e77f-4e38-bd60-e973f19ec88f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:17.696641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe0582-46d3-4b7d-b9f9-d24bae9ad908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:17.697916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe0582-46d3-4b7d-b9f9-d24bae9ad908 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:17.698509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe0582-46d3-4b7d-b9f9-d24bae9ad908 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:17.698790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe0582-46d3-4b7d-b9f9-d24bae9ad908 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:17.705023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:17.708531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe0582-46d3-4b7d-b9f9-d24bae9ad908 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:17.710104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe0582-46d3-4b7d-b9f9-d24bae9ad908 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-aafe0582-46d3-4b7d-b9f9-d24bae9ad908 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:17.727623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:17.737859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:17.739895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:17.739895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:17.739895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:17.739895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:17.739895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:17.739895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:17.739895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:17.739895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:17.739895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 342/671] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:17.739895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe0582-46d3-4b7d-b9f9-d24bae9ad908 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:17.752209 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:17.802304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e02d63-e77f-4e38-bd60-e973f19ec88f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:17.841076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c45e86-fd5d-4fb5-b4f4-0db7747ad759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:17.841741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c45e86-fd5d-4fb5-b4f4-0db7747ad759 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:17.842025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c45e86-fd5d-4fb5-b4f4-0db7747ad759 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:17.842297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c45e86-fd5d-4fb5-b4f4-0db7747ad759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:17.843154 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c45e86-fd5d-4fb5-b4f4-0db7747ad759 None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=78b67db080874f3ebb508239c7a8ccf8) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:17.843466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c45e86-fd5d-4fb5-b4f4-0db7747ad759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:17.844962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c45e86-fd5d-4fb5-b4f4-0db7747ad759 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:17.853776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 330/672] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:17 2022] GET /identity/v3/role_assignments?user.id=78b67db080874f3ebb508239c7a8ccf8&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:17.855046 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c45e86-fd5d-4fb5-b4f4-0db7747ad759 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:17.880611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe0582-46d3-4b7d-b9f9-d24bae9ad908 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:17.949531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3cae-5156-4d77-aa61-cc303b3e5301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:17.951433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3cae-5156-4d77-aa61-cc303b3e5301 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:17.952287 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3cae-5156-4d77-aa61-cc303b3e5301 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:17.953224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3cae-5156-4d77-aa61-cc303b3e5301 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:17.955064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce3cae-5156-4d77-aa61-cc303b3e5301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:17.955994 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce3cae-5156-4d77-aa61-cc303b3e5301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:17.959115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce3cae-5156-4d77-aa61-cc303b3e5301 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:17.985414 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 343/673] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:17.987375 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3cae-5156-4d77-aa61-cc303b3e5301 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.019210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:18.034113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c45e86-fd5d-4fb5-b4f4-0db7747ad759 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.089437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff83ed-8a3a-47f4-8c70-cfaa02919272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.092267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff83ed-8a3a-47f4-8c70-cfaa02919272 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.092812 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff83ed-8a3a-47f4-8c70-cfaa02919272 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.093244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff83ed-8a3a-47f4-8c70-cfaa02919272 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.111422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff83ed-8a3a-47f4-8c70-cfaa02919272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:18.117703 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-a0ff83ed-8a3a-47f4-8c70-cfaa02919272 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:18.119850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:18.121434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 331/674] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:18.123121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f957adc-6eab-4db2-9fc2-3c1f274a81d2 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.129891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f957adc-6eab-4db2-9fc2-3c1f274a81d2 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.129891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f957adc-6eab-4db2-9fc2-3c1f274a81d2 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.129891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 332/675] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:18.129891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f957adc-6eab-4db2-9fc2-3c1f274a81d2 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.195614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3cae-5156-4d77-aa61-cc303b3e5301 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.241777 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b401b-f272-4bf4-8732-f14bd73d0e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.242499 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b401b-f272-4bf4-8732-f14bd73d0e0a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.242783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b401b-f272-4bf4-8732-f14bd73d0e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.243059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b401b-f272-4bf4-8732-f14bd73d0e0a None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.243850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b401b-f272-4bf4-8732-f14bd73d0e0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:18.244155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b401b-f272-4bf4-8732-f14bd73d0e0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:18.245669 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b401b-f272-4bf4-8732-f14bd73d0e0a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:18.276928 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 344/676] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:18.276928 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fa68f-714d-4a59-a312-cbccf8d8272f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.276928 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fa68f-714d-4a59-a312-cbccf8d8272f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.276928 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fa68f-714d-4a59-a312-cbccf8d8272f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.287810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f957adc-6eab-4db2-9fc2-3c1f274a81d2 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.295850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-871fa68f-714d-4a59-a312-cbccf8d8272f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:18.297274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-871fa68f-714d-4a59-a312-cbccf8d8272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:18.334750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e1065-d7f7-4a35-9561-5d19539718b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.335936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e1065-d7f7-4a35-9561-5d19539718b5 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.336413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e1065-d7f7-4a35-9561-5d19539718b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.336916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e1065-d7f7-4a35-9561-5d19539718b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.347107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e1065-d7f7-4a35-9561-5d19539718b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:18.349220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e1065-d7f7-4a35-9561-5d19539718b5 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-500e1065-d7f7-4a35-9561-5d19539718b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:18.367424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:18.369183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:18.371442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:18.371442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:18.371442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:18.371442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:18.371442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:18.371442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:18.371442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:18.371442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:18.371442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 333/677] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:18.371442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e1065-d7f7-4a35-9561-5d19539718b5 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.389854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 345/678] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:18.391918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:18.402333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871fa68f-714d-4a59-a312-cbccf8d8272f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.496860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e1065-d7f7-4a35-9561-5d19539718b5 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.511400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:18.551519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e810ca0-6dd6-4703-b031-2b91234c7718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.552705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e810ca0-6dd6-4703-b031-2b91234c7718 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.553207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e810ca0-6dd6-4703-b031-2b91234c7718 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.553710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e810ca0-6dd6-4703-b031-2b91234c7718 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.555107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e810ca0-6dd6-4703-b031-2b91234c7718 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:18.555650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e810ca0-6dd6-4703-b031-2b91234c7718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:18.558230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e810ca0-6dd6-4703-b031-2b91234c7718 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:18.566988 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871fa68f-714d-4a59-a312-cbccf8d8272f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.576731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 334/679] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:18.578230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e810ca0-6dd6-4703-b031-2b91234c7718 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.626083 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d04d3-a31b-4240-98e7-99525ad65d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.627115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d04d3-a31b-4240-98e7-99525ad65d03 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.627397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d04d3-a31b-4240-98e7-99525ad65d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.627676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d04d3-a31b-4240-98e7-99525ad65d03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.628889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d04d3-a31b-4240-98e7-99525ad65d03 None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=a697d1e7e42048a8a1fb997bcc7b3fab) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:18.629377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d04d3-a31b-4240-98e7-99525ad65d03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:18.630873 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d04d3-a31b-4240-98e7-99525ad65d03 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:18.648526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 346/680] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/role_assignments?user.id=a697d1e7e42048a8a1fb997bcc7b3fab&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:18.648526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d04d3-a31b-4240-98e7-99525ad65d03 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.694715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:18.725981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e810ca0-6dd6-4703-b031-2b91234c7718 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.775862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4815a6-7aac-46f5-aa94-a22db1a3d2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.776261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d04d3-a31b-4240-98e7-99525ad65d03 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:18.776975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4815a6-7aac-46f5-aa94-a22db1a3d2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.777232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4815a6-7aac-46f5-aa94-a22db1a3d2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.777685 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4815a6-7aac-46f5-aa94-a22db1a3d2f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.779158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4815a6-7aac-46f5-aa94-a22db1a3d2f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:18.780526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4815a6-7aac-46f5-aa94-a22db1a3d2f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:18.782055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4815a6-7aac-46f5-aa94-a22db1a3d2f8 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:18.803751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 335/681] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:18.804870 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4815a6-7aac-46f5-aa94-a22db1a3d2f8 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.809623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:18.838733 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91689b2-c4ef-404d-bb70-9d17af1a068b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:18.839921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91689b2-c4ef-404d-bb70-9d17af1a068b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:18.840711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91689b2-c4ef-404d-bb70-9d17af1a068b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:18.841168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91689b2-c4ef-404d-bb70-9d17af1a068b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:18.856995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91689b2-c4ef-404d-bb70-9d17af1a068b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-d91689b2-c4ef-404d-bb70-9d17af1a068b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:18.867721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:18.868866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:18.869710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:18.869710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:18.869710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:18.869710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:18.869710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:18.869710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:18.869710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:18.869710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:18.869710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:18.869710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:18.869710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:18.869710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:18.876415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 347/682] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:18.876415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91689b2-c4ef-404d-bb70-9d17af1a068b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:18.979952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4815a6-7aac-46f5-aa94-a22db1a3d2f8 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.041765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbceb5a-a083-4f8d-88e1-4e6ed56ad905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.042740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbceb5a-a083-4f8d-88e1-4e6ed56ad905 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.043083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbceb5a-a083-4f8d-88e1-4e6ed56ad905 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.043437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbceb5a-a083-4f8d-88e1-4e6ed56ad905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.046820 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:19.057168 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbceb5a-a083-4f8d-88e1-4e6ed56ad905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.059921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbceb5a-a083-4f8d-88e1-4e6ed56ad905 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.072358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91689b2-c4ef-404d-bb70-9d17af1a068b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-9cbceb5a-a083-4f8d-88e1-4e6ed56ad905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:19.086034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:19.087740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:19.089456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:19.089456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:19.089456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:19.089456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:19.089456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 336/683] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:19.089456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbceb5a-a083-4f8d-88e1-4e6ed56ad905 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.128819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ec955-9ed4-4ea1-9b8d-3254aac04a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.129820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ec955-9ed4-4ea1-9b8d-3254aac04a4c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.130103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ec955-9ed4-4ea1-9b8d-3254aac04a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.130382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ec955-9ed4-4ea1-9b8d-3254aac04a4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.131837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ec955-9ed4-4ea1-9b8d-3254aac04a4c None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=8a120d9e7e204dc5b972fd9402897b41) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:19.132158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ec955-9ed4-4ea1-9b8d-3254aac04a4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.134200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ec955-9ed4-4ea1-9b8d-3254aac04a4c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.143637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 348/684] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:18 2022] GET /identity/v3/role_assignments?user.id=8a120d9e7e204dc5b972fd9402897b41&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:19.144559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ec955-9ed4-4ea1-9b8d-3254aac04a4c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.205372 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:19.285177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ec955-9ed4-4ea1-9b8d-3254aac04a4c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.331206 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e98e8-bbba-43a2-ad6b-151b0f0312b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.332205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e98e8-bbba-43a2-ad6b-151b0f0312b5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.332712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e98e8-bbba-43a2-ad6b-151b0f0312b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.333170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e98e8-bbba-43a2-ad6b-151b0f0312b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.334329 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e98e8-bbba-43a2-ad6b-151b0f0312b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:19.334849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e98e8-bbba-43a2-ad6b-151b0f0312b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.337405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e98e8-bbba-43a2-ad6b-151b0f0312b5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.354717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 349/685] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:19.361823 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e98e8-bbba-43a2-ad6b-151b0f0312b5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.437055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbceb5a-a083-4f8d-88e1-4e6ed56ad905 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.515041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e98e8-bbba-43a2-ad6b-151b0f0312b5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.516194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:19.517163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6795b68-bced-402e-9204-1db4e973515b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.519030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6795b68-bced-402e-9204-1db4e973515b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.519569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6795b68-bced-402e-9204-1db4e973515b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.520341 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6795b68-bced-402e-9204-1db4e973515b None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.521489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6795b68-bced-402e-9204-1db4e973515b None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:19.522422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6795b68-bced-402e-9204-1db4e973515b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.525528 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6795b68-bced-402e-9204-1db4e973515b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.552118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 337/686] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:19.558379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6795b68-bced-402e-9204-1db4e973515b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.573481 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c8d7d-9c3a-4fb3-be20-e48200f5746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.575023 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c8d7d-9c3a-4fb3-be20-e48200f5746e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.575834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c8d7d-9c3a-4fb3-be20-e48200f5746e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.575834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c8d7d-9c3a-4fb3-be20-e48200f5746e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.585741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c8d7d-9c3a-4fb3-be20-e48200f5746e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.587465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c8d7d-9c3a-4fb3-be20-e48200f5746e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-405c8d7d-9c3a-4fb3-be20-e48200f5746e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:19.620916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:19.622917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:19.625875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:19.625875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:19.625875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:19.625875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:19.625875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:19.625875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:19.625875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:19.625875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:19.625875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:19.625875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:19.625875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 350/687] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:19.633739 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c8d7d-9c3a-4fb3-be20-e48200f5746e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.654133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:19.717883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6795b68-bced-402e-9204-1db4e973515b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.758616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24b54b-6f1a-4ee0-a61a-cf9fb5ce9175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.759731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24b54b-6f1a-4ee0-a61a-cf9fb5ce9175 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.760160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24b54b-6f1a-4ee0-a61a-cf9fb5ce9175 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.760677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24b54b-6f1a-4ee0-a61a-cf9fb5ce9175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.767799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c8d7d-9c3a-4fb3-be20-e48200f5746e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:19.774469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24b54b-6f1a-4ee0-a61a-cf9fb5ce9175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.776317 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24b54b-6f1a-4ee0-a61a-cf9fb5ce9175 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-ae24b54b-6f1a-4ee0-a61a-cf9fb5ce9175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:19.796625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:19.797680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:19.798687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:19.798687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:19.798687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:19.798687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:19.800718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 338/688] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:19.810801 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24b54b-6f1a-4ee0-a61a-cf9fb5ce9175 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.811174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea2ef0-97b2-4d78-ad5c-27777173b8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:19.811875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea2ef0-97b2-4d78-ad5c-27777173b8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:19.812511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea2ef0-97b2-4d78-ad5c-27777173b8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:19.812640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea2ef0-97b2-4d78-ad5c-27777173b8c3 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:19.816533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea2ef0-97b2-4d78-ad5c-27777173b8c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:19.816533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea2ef0-97b2-4d78-ad5c-27777173b8c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:19.816533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea2ef0-97b2-4d78-ad5c-27777173b8c3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:19.835858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 351/689] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:19.845798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea2ef0-97b2-4d78-ad5c-27777173b8c3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:19.906024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:20.107351 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea2ef0-97b2-4d78-ad5c-27777173b8c3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:20.115060 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24b54b-6f1a-4ee0-a61a-cf9fb5ce9175 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:20.118611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:20.227865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b9f33-aff7-4788-b55f-39b88db08bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:20.229025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b9f33-aff7-4788-b55f-39b88db08bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:20.229536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b9f33-aff7-4788-b55f-39b88db08bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:20.230029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b9f33-aff7-4788-b55f-39b88db08bc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:20.239913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e4992-af39-423b-a73b-f0923a657763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:20.241127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e4992-af39-423b-a73b-f0923a657763 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:20.241640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e4992-af39-423b-a73b-f0923a657763 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:20.242146 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e4992-af39-423b-a73b-f0923a657763 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:20.243651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e4992-af39-423b-a73b-f0923a657763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:20.244200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e4992-af39-423b-a73b-f0923a657763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:20.246961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e4992-af39-423b-a73b-f0923a657763 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:20.301872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 339/690] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 492 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:20.311480 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e4992-af39-423b-a73b-f0923a657763 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:20.486235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b9f33-aff7-4788-b55f-39b88db08bc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:20.488827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b9f33-aff7-4788-b55f-39b88db08bc5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:20.560401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e4992-af39-423b-a73b-f0923a657763 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:20.587312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:20.654750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7d769-15d7-40f3-ac68-be4ea1bd8042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:20.656573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7d769-15d7-40f3-ac68-be4ea1bd8042 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:20.657279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7d769-15d7-40f3-ac68-be4ea1bd8042 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:20.657968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7d769-15d7-40f3-ac68-be4ea1bd8042 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:20.691280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7d769-15d7-40f3-ac68-be4ea1bd8042 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:20.695539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7d769-15d7-40f3-ac68-be4ea1bd8042 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-22f7d769-15d7-40f3-ac68-be4ea1bd8042 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:20.729224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:20.732140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:20.734364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:20.734364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:20.734364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:20.734364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:20.734364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:20.734364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:20.734364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:20.734364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:20.734364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:20.734364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:20.737101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 340/691] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:20.743343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7d769-15d7-40f3-ac68-be4ea1bd8042 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:20.818440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 352/692] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 973 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:20:20.819241 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:20.834291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b9f33-aff7-4788-b55f-39b88db08bc5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:20.923867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7d769-15d7-40f3-ac68-be4ea1bd8042 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:20.962764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875c38e-d398-4483-b2c2-b674706bdc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:20.963542 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875c38e-d398-4483-b2c2-b674706bdc42 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:20.963978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875c38e-d398-4483-b2c2-b674706bdc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:20.964312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875c38e-d398-4483-b2c2-b674706bdc42 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:20.965490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875c38e-d398-4483-b2c2-b674706bdc42 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:20.965900 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875c38e-d398-4483-b2c2-b674706bdc42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:20.967489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875c38e-d398-4483-b2c2-b674706bdc42 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:20.993319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 341/693] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jul 21 22:20:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:20.997323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8b9f4-2b74-4e6d-9f10-81a48f76201f None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:20.997754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8b9f4-2b74-4e6d-9f10-81a48f76201f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:20.998455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8b9f4-2b74-4e6d-9f10-81a48f76201f None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:21.000620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 342/694] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:21.002142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8b9f4-2b74-4e6d-9f10-81a48f76201f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:21.071938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b9f33-aff7-4788-b55f-39b88db08bc5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:21.115848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:21.117157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:21.137789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a0d5e-3521-4dd0-a77b-29dea6dba051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:21.139508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a0d5e-3521-4dd0-a77b-29dea6dba051 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:21.140237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a0d5e-3521-4dd0-a77b-29dea6dba051 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:21.141000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a0d5e-3521-4dd0-a77b-29dea6dba051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:21.143604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8b9f4-2b74-4e6d-9f10-81a48f76201f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:21.161109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a0d5e-3521-4dd0-a77b-29dea6dba051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:21.164212 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a0d5e-3521-4dd0-a77b-29dea6dba051 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:21.188654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c8094-bbbd-4d0d-b3ba-2cf93c1add8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:21.189404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c8094-bbbd-4d0d-b3ba-2cf93c1add8d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:21.189753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c8094-bbbd-4d0d-b3ba-2cf93c1add8d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:21.190094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c8094-bbbd-4d0d-b3ba-2cf93c1add8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-2f7a0d5e-3521-4dd0-a77b-29dea6dba051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:21.200026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:21.201762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:21.203545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:21.203545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:21.203545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:21.203545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:21.214213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 353/695] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:21.217050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e9fdb-78e4-49ff-8774-1fdcde9ab28c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:21.217853 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e9fdb-78e4-49ff-8774-1fdcde9ab28c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:21.218625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e9fdb-78e4-49ff-8774-1fdcde9ab28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:21.248395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-823e9fdb-78e4-49ff-8774-1fdcde9ab28c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:21.250895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-823e9fdb-78e4-49ff-8774-1fdcde9ab28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:21.272672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c8094-bbbd-4d0d-b3ba-2cf93c1add8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:21.274551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c8094-bbbd-4d0d-b3ba-2cf93c1add8d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:21.378359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 354/696] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:21.380341 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e9fdb-78e4-49ff-8774-1fdcde9ab28c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:21.387252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:21.411000 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 343/697] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:20:21.418679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c8094-bbbd-4d0d-b3ba-2cf93c1add8d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:21.520409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e9fdb-78e4-49ff-8774-1fdcde9ab28c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:21.523180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:21.547969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c8094-bbbd-4d0d-b3ba-2cf93c1add8d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:21.558151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bb7c0-1f6e-4b9b-9420-fb663fc812ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:21.558833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bb7c0-1f6e-4b9b-9420-fb663fc812ce None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:21.559312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bb7c0-1f6e-4b9b-9420-fb663fc812ce None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:21.559594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bb7c0-1f6e-4b9b-9420-fb663fc812ce None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:21.560749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bb7c0-1f6e-4b9b-9420-fb663fc812ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:21.561062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bb7c0-1f6e-4b9b-9420-fb663fc812ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:21.562801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bb7c0-1f6e-4b9b-9420-fb663fc812ce None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:21.578745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 355/698] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:21.579732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bb7c0-1f6e-4b9b-9420-fb663fc812ce None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:21.588248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210b225-2f5f-47de-9cef-756d075bbaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:21.590216 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210b225-2f5f-47de-9cef-756d075bbaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:21.590749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210b225-2f5f-47de-9cef-756d075bbaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:21.591193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210b225-2f5f-47de-9cef-756d075bbaa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:21.603277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210b225-2f5f-47de-9cef-756d075bbaa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:21.605080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210b225-2f5f-47de-9cef-756d075bbaa8 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-0210b225-2f5f-47de-9cef-756d075bbaa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:21.625931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:21.627109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:21.628131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:21.628131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:21.628131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:21.628131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:21.634619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 344/699] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:21.635786 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210b225-2f5f-47de-9cef-756d075bbaa8 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:21.675548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:21.702423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bb7c0-1f6e-4b9b-9420-fb663fc812ce None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:21.746251 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacb2b3-57c7-464a-a5c4-05ce6ede375d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:21.747861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacb2b3-57c7-464a-a5c4-05ce6ede375d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:21.748349 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacb2b3-57c7-464a-a5c4-05ce6ede375d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:21.748865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacb2b3-57c7-464a-a5c4-05ce6ede375d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:21.764394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210b225-2f5f-47de-9cef-756d075bbaa8 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:21.765228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dacb2b3-57c7-464a-a5c4-05ce6ede375d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:21.768073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dacb2b3-57c7-464a-a5c4-05ce6ede375d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-3dacb2b3-57c7-464a-a5c4-05ce6ede375d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:21.792178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:21.793831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:21.794842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:21.794842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:21.794842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:21.794842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:21.797528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 356/700] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:21.798512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacb2b3-57c7-464a-a5c4-05ce6ede375d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:21.809638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ceb7e9-bff2-43be-a427-876b3deca658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:21.810548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ceb7e9-bff2-43be-a427-876b3deca658 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:21.810939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ceb7e9-bff2-43be-a427-876b3deca658 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:21.811653 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ceb7e9-bff2-43be-a427-876b3deca658 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:21.819054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ceb7e9-bff2-43be-a427-876b3deca658 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:21.820772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ceb7e9-bff2-43be-a427-876b3deca658 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:21.823170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-96ceb7e9-bff2-43be-a427-876b3deca658 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:21.841089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:21.842255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:21.843264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:21.843264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:21.843264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:21.843264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:21.843264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:21.843264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:21.843264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:21.843264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:21.843718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 345/701] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:21.844922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ceb7e9-bff2-43be-a427-876b3deca658 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:21.926852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacb2b3-57c7-464a-a5c4-05ce6ede375d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:21.967985 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc2da2-0461-40a3-a3b1-53ff6f9ca6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:21.968975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc2da2-0461-40a3-a3b1-53ff6f9ca6de None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:21.969426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc2da2-0461-40a3-a3b1-53ff6f9ca6de None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:21.969862 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc2da2-0461-40a3-a3b1-53ff6f9ca6de None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:21.970744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc2da2-0461-40a3-a3b1-53ff6f9ca6de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:21.971299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc2da2-0461-40a3-a3b1-53ff6f9ca6de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:21.972908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc2da2-0461-40a3-a3b1-53ff6f9ca6de None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:21.975326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:21.987899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ceb7e9-bff2-43be-a427-876b3deca658 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:21.994231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 357/702] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:21.995402 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc2da2-0461-40a3-a3b1-53ff6f9ca6de None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:22.040620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33e626-746c-4611-9560-06f4e73b8681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:22.041326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33e626-746c-4611-9560-06f4e73b8681 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.041614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33e626-746c-4611-9560-06f4e73b8681 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.041894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33e626-746c-4611-9560-06f4e73b8681 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.043680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33e626-746c-4611-9560-06f4e73b8681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:22.043910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33e626-746c-4611-9560-06f4e73b8681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:22.045381 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33e626-746c-4611-9560-06f4e73b8681 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:22.064574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 346/703] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:22.065722 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33e626-746c-4611-9560-06f4e73b8681 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:22.115645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:22.147060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc2da2-0461-40a3-a3b1-53ff6f9ca6de None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:22.201671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356cefa-7a1d-44a3-83cc-ce63268fb244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:22.202358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356cefa-7a1d-44a3-83cc-ce63268fb244 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.202797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356cefa-7a1d-44a3-83cc-ce63268fb244 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.203091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356cefa-7a1d-44a3-83cc-ce63268fb244 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.204606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356cefa-7a1d-44a3-83cc-ce63268fb244 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:22.204987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356cefa-7a1d-44a3-83cc-ce63268fb244 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:22.206798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356cefa-7a1d-44a3-83cc-ce63268fb244 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:22.220724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33e626-746c-4611-9560-06f4e73b8681 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:22.222820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 358/704] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:22.224074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356cefa-7a1d-44a3-83cc-ce63268fb244 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:22.295784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c5230-47d6-4d49-9b5e-60ffc62fc2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:22.297443 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c5230-47d6-4d49-9b5e-60ffc62fc2ac None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.298201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c5230-47d6-4d49-9b5e-60ffc62fc2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.298899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c5230-47d6-4d49-9b5e-60ffc62fc2ac None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.300694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c5230-47d6-4d49-9b5e-60ffc62fc2ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:22.303637 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c5230-47d6-4d49-9b5e-60ffc62fc2ac None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:22.341398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 347/705] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:20:22 2022] POST /identity/v3/users => generated 254 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:22.343686 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c5230-47d6-4d49-9b5e-60ffc62fc2ac None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:22.367587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:22.391469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356cefa-7a1d-44a3-83cc-ce63268fb244 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:22.445310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8ac95-cb68-44f0-b3c3-a396e87ff602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:22.446134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8ac95-cb68-44f0-b3c3-a396e87ff602 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.446545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8ac95-cb68-44f0-b3c3-a396e87ff602 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.446830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8ac95-cb68-44f0-b3c3-a396e87ff602 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.451755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:22.473128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8ac95-cb68-44f0-b3c3-a396e87ff602 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:22.475288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8ac95-cb68-44f0-b3c3-a396e87ff602 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-a7d8ac95-cb68-44f0-b3c3-a396e87ff602 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:22.496967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:22.498154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:22.499054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:22.499054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:22.499054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:22.499054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:22.499054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:22.499054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:22.499054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:22.499054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:22.499054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 359/706] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:22.500020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382ab14-aebb-4b8f-90d8-c9d1aed84236 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.501316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382ab14-aebb-4b8f-90d8-c9d1aed84236 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.501316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382ab14-aebb-4b8f-90d8-c9d1aed84236 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.503591 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 360/707] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:22.517795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382ab14-aebb-4b8f-90d8-c9d1aed84236 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:22.563358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c5230-47d6-4d49-9b5e-60ffc62fc2ac None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:22.620214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed1a24-49d7-43fb-b0ed-ecc6c87ac83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:22.620950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed1a24-49d7-43fb-b0ed-ecc6c87ac83d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.621489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed1a24-49d7-43fb-b0ed-ecc6c87ac83d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.621775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed1a24-49d7-43fb-b0ed-ecc6c87ac83d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.622638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed1a24-49d7-43fb-b0ed-ecc6c87ac83d None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=c77f6ccc7cb84c04ac532007a04a0f5e) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:22.622959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed1a24-49d7-43fb-b0ed-ecc6c87ac83d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:22.624418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed1a24-49d7-43fb-b0ed-ecc6c87ac83d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:22.631534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 348/708] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:22 2022] GET /identity/v3/role_assignments?user.id=c77f6ccc7cb84c04ac532007a04a0f5e&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=f957e09d047b47cfa0891d25b4181263 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:22.633397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16773982-25ae-4049-bacf-65787e4c7a95 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.633612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16773982-25ae-4049-bacf-65787e4c7a95 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.633856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16773982-25ae-4049-bacf-65787e4c7a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.661982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-16773982-25ae-4049-bacf-65787e4c7a95 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:22.663314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-16773982-25ae-4049-bacf-65787e4c7a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:22.676836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382ab14-aebb-4b8f-90d8-c9d1aed84236 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:22.757530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404a9ec-5a72-44a6-a4ce-b1c488a1216c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:22.758551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404a9ec-5a72-44a6-a4ce-b1c488a1216c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.759041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404a9ec-5a72-44a6-a4ce-b1c488a1216c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.759588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404a9ec-5a72-44a6-a4ce-b1c488a1216c None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.761854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404a9ec-5a72-44a6-a4ce-b1c488a1216c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:22.762857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404a9ec-5a72-44a6-a4ce-b1c488a1216c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:22.765168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404a9ec-5a72-44a6-a4ce-b1c488a1216c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:22.787462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 349/709] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:22.789805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 361/710] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:22.790371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a6e4a-1ce7-4560-819d-36c8cf425c25 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.790808 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a6e4a-1ce7-4560-819d-36c8cf425c25 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.791213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a6e4a-1ce7-4560-819d-36c8cf425c25 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.800548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 350/711] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:22.802352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cbbdf-0dbf-48ea-8900-f653348e46cc None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:22.803829 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a6e4a-1ce7-4560-819d-36c8cf425c25 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:22.809611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:22.809839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cbbdf-0dbf-48ea-8900-f653348e46cc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:22.810228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cbbdf-0dbf-48ea-8900-f653348e46cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:22.842017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a97cbbdf-0dbf-48ea-8900-f653348e46cc None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:22.844758 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a97cbbdf-0dbf-48ea-8900-f653348e46cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:22.938673 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:22.966059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 362/712] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:22.967515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cbbdf-0dbf-48ea-8900-f653348e46cc None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:22.983759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a6e4a-1ce7-4560-819d-36c8cf425c25 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.034201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef265776-2d65-4df2-a45e-2134ed536791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:23.035317 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef265776-2d65-4df2-a45e-2134ed536791 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.035938 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef265776-2d65-4df2-a45e-2134ed536791 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.036422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef265776-2d65-4df2-a45e-2134ed536791 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.037632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef265776-2d65-4df2-a45e-2134ed536791 None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=8fec30ef0b8c41dca643cfefebfe904b) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:23.038160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef265776-2d65-4df2-a45e-2134ed536791 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:23.039977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef265776-2d65-4df2-a45e-2134ed536791 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:23.050325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 351/713] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:22 2022] GET /identity/v3/role_assignments?user.id=8fec30ef0b8c41dca643cfefebfe904b&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:23.051592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef265776-2d65-4df2-a45e-2134ed536791 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:23.202907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cbbdf-0dbf-48ea-8900-f653348e46cc None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.203561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:23.262324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef265776-2d65-4df2-a45e-2134ed536791 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.317310 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17911933-12ec-49a1-bee4-f98e67e8e9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:23.318504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17911933-12ec-49a1-bee4-f98e67e8e9ab None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.318894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17911933-12ec-49a1-bee4-f98e67e8e9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.319185 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17911933-12ec-49a1-bee4-f98e67e8e9ab None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.319968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17911933-12ec-49a1-bee4-f98e67e8e9ab None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:23.322489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17911933-12ec-49a1-bee4-f98e67e8e9ab None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:23.325442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93efd2-3858-4780-a53c-4d8af3ac49f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:23.326308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93efd2-3858-4780-a53c-4d8af3ac49f0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.326596 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93efd2-3858-4780-a53c-4d8af3ac49f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.326876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93efd2-3858-4780-a53c-4d8af3ac49f0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.327561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93efd2-3858-4780-a53c-4d8af3ac49f0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:23.329083 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93efd2-3858-4780-a53c-4d8af3ac49f0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-17911933-12ec-49a1-bee4-f98e67e8e9ab None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:23.349661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:23.350715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:23.351587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 352/714] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Jul 21 22:20:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:23.361085 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17911933-12ec-49a1-bee4-f98e67e8e9ab None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-0f93efd2-3858-4780-a53c-4d8af3ac49f0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:23.366334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:23.368755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:23.371230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 363/715] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:22 2022] GET /identity/v3/services/network => generated 88 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:23.382461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93efd2-3858-4780-a53c-4d8af3ac49f0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:23.499823 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:23.564381 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17911933-12ec-49a1-bee4-f98e67e8e9ab None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.605549 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:23.613872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763f18e-58a5-4cb6-8004-02295fc08603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:23.615024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763f18e-58a5-4cb6-8004-02295fc08603 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.615525 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763f18e-58a5-4cb6-8004-02295fc08603 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.616033 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763f18e-58a5-4cb6-8004-02295fc08603 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.617050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763f18e-58a5-4cb6-8004-02295fc08603 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:23.617592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763f18e-58a5-4cb6-8004-02295fc08603 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:23.619224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763f18e-58a5-4cb6-8004-02295fc08603 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:23.623538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93efd2-3858-4780-a53c-4d8af3ac49f0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.625794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 353/716] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jul 21 22:20:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:23.643644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763f18e-58a5-4cb6-8004-02295fc08603 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:23.705151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd9af8-084e-48d9-a689-8f6b8045cd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:23.706315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd9af8-084e-48d9-a689-8f6b8045cd95 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.706829 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd9af8-084e-48d9-a689-8f6b8045cd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.707344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd9af8-084e-48d9-a689-8f6b8045cd95 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.708824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd9af8-084e-48d9-a689-8f6b8045cd95 None admin] RBAC: Adding query filter params (name=network) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:23.715991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd9af8-084e-48d9-a689-8f6b8045cd95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:23.718711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd9af8-084e-48d9-a689-8f6b8045cd95 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:23.747467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 364/717] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:23.762306 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd9af8-084e-48d9-a689-8f6b8045cd95 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:23.923147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd9af8-084e-48d9-a689-8f6b8045cd95 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.939437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:23.954086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763f18e-58a5-4cb6-8004-02295fc08603 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:23.989205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef8063-bffc-4ad5-81e1-5762f2f9ddcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:23.990272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef8063-bffc-4ad5-81e1-5762f2f9ddcb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:23.990780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef8063-bffc-4ad5-81e1-5762f2f9ddcb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:23.991237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef8063-bffc-4ad5-81e1-5762f2f9ddcb None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:23.992159 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef8063-bffc-4ad5-81e1-5762f2f9ddcb None admin] RBAC: Adding query filter params (type=network) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:23.992720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef8063-bffc-4ad5-81e1-5762f2f9ddcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:23.994377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef8063-bffc-4ad5-81e1-5762f2f9ddcb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:24.009445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 365/718] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:24.035045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab6e02-f58a-470c-915a-ebc044b13a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:24.036200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab6e02-f58a-470c-915a-ebc044b13a7d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:24.036732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab6e02-f58a-470c-915a-ebc044b13a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:24.037295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab6e02-f58a-470c-915a-ebc044b13a7d None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:24.038643 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab6e02-f58a-470c-915a-ebc044b13a7d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:24.039203 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab6e02-f58a-470c-915a-ebc044b13a7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:24.050232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab6e02-f58a-470c-915a-ebc044b13a7d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:24.061710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 354/719] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jul 21 22:20:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:25.276292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d81c8-2baf-4fb9-aa30-2d1275f398c6 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:25.277693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d81c8-2baf-4fb9-aa30-2d1275f398c6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:25.277693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d81c8-2baf-4fb9-aa30-2d1275f398c6 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:25.279510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 366/720] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:25.285615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d615c34-0588-4562-bba1-cd27d300b14a None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:25.286517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d615c34-0588-4562-bba1-cd27d300b14a None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:25.286517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d615c34-0588-4562-bba1-cd27d300b14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:25.305890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-3d615c34-0588-4562-bba1-cd27d300b14a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:25.307104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3d615c34-0588-4562-bba1-cd27d300b14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:25.371342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:25.389086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 355/721] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:25.422584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d81c8-2baf-4fb9-aa30-2d1275f398c6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:25.594202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d81c8-2baf-4fb9-aa30-2d1275f398c6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:25.666429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e57372-83d2-417b-b36a-3de3efd00c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:25.668516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e57372-83d2-417b-b36a-3de3efd00c93 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:25.669322 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e57372-83d2-417b-b36a-3de3efd00c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:25.670116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e57372-83d2-417b-b36a-3de3efd00c93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:25.702884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e57372-83d2-417b-b36a-3de3efd00c93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b8e57372-83d2-417b-b36a-3de3efd00c93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:25.723487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:25.725395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:25.727700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 367/722] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:25.733958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d615c34-0588-4562-bba1-cd27d300b14a None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:25.864731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a7e3f-6b34-4f02-a6ca-0da8bd572eba None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:25.864731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a7e3f-6b34-4f02-a6ca-0da8bd572eba None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:25.865496 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a7e3f-6b34-4f02-a6ca-0da8bd572eba None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:25.866604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 368/723] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:25.872256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2e6af-7483-4243-8dbf-d1f7de3405c9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:25.872657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2e6af-7483-4243-8dbf-d1f7de3405c9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:25.872893 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2e6af-7483-4243-8dbf-d1f7de3405c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:25.894121 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7ca2e6af-7483-4243-8dbf-d1f7de3405c9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:25.895587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7ca2e6af-7483-4243-8dbf-d1f7de3405c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:25.913267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d615c34-0588-4562-bba1-cd27d300b14a None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:25.961034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:25.995215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253ec9c-a5ca-4be2-b106-10cbdbf8af6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:25.997171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253ec9c-a5ca-4be2-b106-10cbdbf8af6f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:25.998429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253ec9c-a5ca-4be2-b106-10cbdbf8af6f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:25.999218 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253ec9c-a5ca-4be2-b106-10cbdbf8af6f None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.001100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253ec9c-a5ca-4be2-b106-10cbdbf8af6f None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:26.002128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253ec9c-a5ca-4be2-b106-10cbdbf8af6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.004815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253ec9c-a5ca-4be2-b106-10cbdbf8af6f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:26.033037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 356/724] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:26.036650 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 369/725] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:26.047817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253ec9c-a5ca-4be2-b106-10cbdbf8af6f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.061550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2e6af-7483-4243-8dbf-d1f7de3405c9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.184386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:26.288704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253ec9c-a5ca-4be2-b106-10cbdbf8af6f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.320804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2e6af-7483-4243-8dbf-d1f7de3405c9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.372101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcf365-0d33-4dc5-b35a-0f8c1af2f027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.374274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcf365-0d33-4dc5-b35a-0f8c1af2f027 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.375420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcf365-0d33-4dc5-b35a-0f8c1af2f027 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.376893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfcf365-0d33-4dc5-b35a-0f8c1af2f027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.389013 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c89eb-89a2-4772-876b-75377555509a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.390820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c89eb-89a2-4772-876b-75377555509a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.391562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c89eb-89a2-4772-876b-75377555509a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.392316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c89eb-89a2-4772-876b-75377555509a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.404357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfcf365-0d33-4dc5-b35a-0f8c1af2f027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.407256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfcf365-0d33-4dc5-b35a-0f8c1af2f027 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:26.415819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c89eb-89a2-4772-876b-75377555509a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-d83c89eb-89a2-4772-876b-75377555509a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:26.432784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:26.435332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:26.438593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:26.438593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:26.438593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:26.438593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:26.438593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:26.438593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:26.438593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:26.438593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:26.438593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:26.438593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:26.438593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:26.438593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:26.443365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 370/726] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-abfcf365-0d33-4dc5-b35a-0f8c1af2f027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:26.445646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:26.447261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:26.449042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:26.449042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:26.449042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:26.449042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:26.449042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 357/727] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:26.450405 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcf365-0d33-4dc5-b35a-0f8c1af2f027 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.451107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3440bdf-1cf4-4e88-a9d1-d8bdefb2c76e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.451916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3440bdf-1cf4-4e88-a9d1-d8bdefb2c76e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.452712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3440bdf-1cf4-4e88-a9d1-d8bdefb2c76e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.456065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 371/728] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:26 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:26.457886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3440bdf-1cf4-4e88-a9d1-d8bdefb2c76e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.515864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:26.645375 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3440bdf-1cf4-4e88-a9d1-d8bdefb2c76e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.672333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfcf365-0d33-4dc5-b35a-0f8c1af2f027 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.674453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:26.701561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46644b-78d4-493f-8aa5-f6bdd95ea3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.703984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46644b-78d4-493f-8aa5-f6bdd95ea3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.704696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46644b-78d4-493f-8aa5-f6bdd95ea3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.705249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46644b-78d4-493f-8aa5-f6bdd95ea3c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.706877 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46644b-78d4-493f-8aa5-f6bdd95ea3c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:26.707994 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46644b-78d4-493f-8aa5-f6bdd95ea3c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.710882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46644b-78d4-493f-8aa5-f6bdd95ea3c4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:26.720352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a4c6-83a6-49d3-b2f2-cbb2e4e9b2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.721166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a4c6-83a6-49d3-b2f2-cbb2e4e9b2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.721542 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a4c6-83a6-49d3-b2f2-cbb2e4e9b2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.721892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a4c6-83a6-49d3-b2f2-cbb2e4e9b2c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.722734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82a4c6-83a6-49d3-b2f2-cbb2e4e9b2c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:26.723116 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82a4c6-83a6-49d3-b2f2-cbb2e4e9b2c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.725462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82a4c6-83a6-49d3-b2f2-cbb2e4e9b2c0 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:26.733243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 372/729] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:26.736670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6bdc8-87d3-4724-8d9b-6717884a1918 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.737826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6bdc8-87d3-4724-8d9b-6717884a1918 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.738023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 358/730] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:26.738768 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6bdc8-87d3-4724-8d9b-6717884a1918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.739002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a4c6-83a6-49d3-b2f2-cbb2e4e9b2c0 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.776947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-99f6bdc8-87d3-4724-8d9b-6717884a1918 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:26.779274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-99f6bdc8-87d3-4724-8d9b-6717884a1918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:26.909176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 373/731] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:26.911206 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6bdc8-87d3-4724-8d9b-6717884a1918 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:26.919687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a4c6-83a6-49d3-b2f2-cbb2e4e9b2c0 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:26.947179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:26.978392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa379d8-40ba-41ac-9c21-0d36edcdac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:26.979288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa379d8-40ba-41ac-9c21-0d36edcdac35 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:26.980545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa379d8-40ba-41ac-9c21-0d36edcdac35 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:26.980545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa379d8-40ba-41ac-9c21-0d36edcdac35 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:26.990986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa379d8-40ba-41ac-9c21-0d36edcdac35 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:26.993348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa379d8-40ba-41ac-9c21-0d36edcdac35 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-efa379d8-40ba-41ac-9c21-0d36edcdac35 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:27.018104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:27.020104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:27.022536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:27.022536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:27.022536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:27.022536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:27.022536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:27.022536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:27.022536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:27.022536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:27.022536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:27.022536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:27.022536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 359/732] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 22:20:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:27.022536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa379d8-40ba-41ac-9c21-0d36edcdac35 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:27.068837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:27.110909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6bdc8-87d3-4724-8d9b-6717884a1918 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:27.166339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b509d-a88b-46c7-8ad4-c707cc398f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:27.168312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b509d-a88b-46c7-8ad4-c707cc398f3b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.168841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b509d-a88b-46c7-8ad4-c707cc398f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.169000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b509d-a88b-46c7-8ad4-c707cc398f3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.179741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa379d8-40ba-41ac-9c21-0d36edcdac35 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:27.182098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b509d-a88b-46c7-8ad4-c707cc398f3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:27.184311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b509d-a88b-46c7-8ad4-c707cc398f3b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-a16b509d-a88b-46c7-8ad4-c707cc398f3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:27.205538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:27.206554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:27.207454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:27.207454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:27.207454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:27.207454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:27.207691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 374/733] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:27.212817 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b509d-a88b-46c7-8ad4-c707cc398f3b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:27.238425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a444a-2cad-4dee-b2ec-6fff14db26cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:27.239312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a444a-2cad-4dee-b2ec-6fff14db26cf None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.239597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a444a-2cad-4dee-b2ec-6fff14db26cf None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.239871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a444a-2cad-4dee-b2ec-6fff14db26cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.252117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a444a-2cad-4dee-b2ec-6fff14db26cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-822a444a-2cad-4dee-b2ec-6fff14db26cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:27.259774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:27.261019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:27.262413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 360/734] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:27.263948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37404e5-fbb0-406b-9784-5bef14c00322 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.263948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37404e5-fbb0-406b-9784-5bef14c00322 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.263948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37404e5-fbb0-406b-9784-5bef14c00322 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.265190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 361/735] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:27.270052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37404e5-fbb0-406b-9784-5bef14c00322 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:27.333529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:27.368232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b509d-a88b-46c7-8ad4-c707cc398f3b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:27.370112 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:27.391127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37404e5-fbb0-406b-9784-5bef14c00322 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:27.412131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ca807-6724-4ad2-a4f0-369110775029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:27.412840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ca807-6724-4ad2-a4f0-369110775029 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.413131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ca807-6724-4ad2-a4f0-369110775029 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.413404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ca807-6724-4ad2-a4f0-369110775029 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.414255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ca807-6724-4ad2-a4f0-369110775029 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:27.414562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ca807-6724-4ad2-a4f0-369110775029 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:27.416033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ca807-6724-4ad2-a4f0-369110775029 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:27.435843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 375/736] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jul 21 22:20:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:27.437188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d2617-8d12-479e-ba56-aaff6329521c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.437475 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d2617-8d12-479e-ba56-aaff6329521c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.437746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d2617-8d12-479e-ba56-aaff6329521c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.442988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc23c2-7fde-40b3-8e1d-740e21d2018a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:27.444136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc23c2-7fde-40b3-8e1d-740e21d2018a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.444687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc23c2-7fde-40b3-8e1d-740e21d2018a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.445187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc23c2-7fde-40b3-8e1d-740e21d2018a None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.446560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc23c2-7fde-40b3-8e1d-740e21d2018a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:27.447087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc23c2-7fde-40b3-8e1d-740e21d2018a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:27.450400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc23c2-7fde-40b3-8e1d-740e21d2018a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:27.455496 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b68d2617-8d12-479e-ba56-aaff6329521c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:27.456728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b68d2617-8d12-479e-ba56-aaff6329521c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:27.470140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 362/737] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:27.471133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc23c2-7fde-40b3-8e1d-740e21d2018a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:27.536634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 376/738] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:27.537975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554238d5-0d20-4fad-b11f-0e0862952fff None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.538261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554238d5-0d20-4fad-b11f-0e0862952fff None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.538533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554238d5-0d20-4fad-b11f-0e0862952fff None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.539805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 377/739] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:27.540520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554238d5-0d20-4fad-b11f-0e0862952fff None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:27.600112 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc23c2-7fde-40b3-8e1d-740e21d2018a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:27.607987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:27.639814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592093-318f-411a-b504-78023c0325f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:27.640526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592093-318f-411a-b504-78023c0325f7 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.640818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592093-318f-411a-b504-78023c0325f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.641100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592093-318f-411a-b504-78023c0325f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.641866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3592093-318f-411a-b504-78023c0325f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:27.642182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3592093-318f-411a-b504-78023c0325f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:27.643577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3592093-318f-411a-b504-78023c0325f7 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:27.654011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 363/740] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:27.654884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592093-318f-411a-b504-78023c0325f7 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:27.656650 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554238d5-0d20-4fad-b11f-0e0862952fff None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:27.666717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:27.702953 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d4b3a-13df-4808-80ac-379d31cb06cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:27.703569 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d4b3a-13df-4808-80ac-379d31cb06cc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.703862 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d4b3a-13df-4808-80ac-379d31cb06cc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.704124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d4b3a-13df-4808-80ac-379d31cb06cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.783447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592093-318f-411a-b504-78023c0325f7 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:27.795411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d4b3a-13df-4808-80ac-379d31cb06cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:27.797956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d4b3a-13df-4808-80ac-379d31cb06cc None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:27.831811 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19805763-87f8-4e2d-b3f3-e970fcb686d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:27.832653 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19805763-87f8-4e2d-b3f3-e970fcb686d9 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.832955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19805763-87f8-4e2d-b3f3-e970fcb686d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.833211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19805763-87f8-4e2d-b3f3-e970fcb686d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.841798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19805763-87f8-4e2d-b3f3-e970fcb686d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:27.843242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19805763-87f8-4e2d-b3f3-e970fcb686d9 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-19805763-87f8-4e2d-b3f3-e970fcb686d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:27.858292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:27.859316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:27.860293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:27.860293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:27.860293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:27.860293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:27.860293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:27.860293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:27.860293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:27.860293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:27.860293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:27.860293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:27.860293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 364/741] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 22:20:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:27.860939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909f23e-3265-4510-a4b2-533325b5babd None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:27.861064 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909f23e-3265-4510-a4b2-533325b5babd None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:27.861339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909f23e-3265-4510-a4b2-533325b5babd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:27.879706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f909f23e-3265-4510-a4b2-533325b5babd None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:27.881173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f909f23e-3265-4510-a4b2-533325b5babd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:27.891108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:27.947969 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 378/742] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:20:27.949445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d4b3a-13df-4808-80ac-379d31cb06cc None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:27.951998 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:27.960660 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 365/743] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:20:27.961513 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909f23e-3265-4510-a4b2-533325b5babd None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:28.078816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d4b3a-13df-4808-80ac-379d31cb06cc None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:28.112106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909f23e-3265-4510-a4b2-533325b5babd None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:28.117709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c80103-196c-44dd-8441-b410f2047033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:28.118482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c80103-196c-44dd-8441-b410f2047033 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:28.118842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c80103-196c-44dd-8441-b410f2047033 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:28.119189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c80103-196c-44dd-8441-b410f2047033 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:28.120020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c80103-196c-44dd-8441-b410f2047033 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:28.121544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c80103-196c-44dd-8441-b410f2047033 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:28.138651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 379/744] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:20:27 2022] POST /identity/v3/services => generated 240 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:28.139507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c80103-196c-44dd-8441-b410f2047033 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:28.164695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:28.185566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768bf24-6571-4853-ac75-42013f0cd5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:28.187141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768bf24-6571-4853-ac75-42013f0cd5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:28.187815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768bf24-6571-4853-ac75-42013f0cd5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:28.188187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768bf24-6571-4853-ac75-42013f0cd5a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:28.209464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768bf24-6571-4853-ac75-42013f0cd5a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:28.211747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768bf24-6571-4853-ac75-42013f0cd5a8 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-7768bf24-6571-4853-ac75-42013f0cd5a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:28.237425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:28.238449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:28.239367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:28.239367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:28.239367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:28.239367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:28.239367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 366/745] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:28.239657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768bf24-6571-4853-ac75-42013f0cd5a8 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:28.298672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:28.326871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c80103-196c-44dd-8441-b410f2047033 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:28.375164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6859f01-502c-4f90-b158-b67f88934c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:28.376171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6859f01-502c-4f90-b158-b67f88934c25 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:28.376712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6859f01-502c-4f90-b158-b67f88934c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:28.377213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6859f01-502c-4f90-b158-b67f88934c25 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:28.378867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6859f01-502c-4f90-b158-b67f88934c25 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:28.379433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6859f01-502c-4f90-b158-b67f88934c25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:28.382247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6859f01-502c-4f90-b158-b67f88934c25 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:28.382999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768bf24-6571-4853-ac75-42013f0cd5a8 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:28.413920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 380/746] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 22:20:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:28.413920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6859f01-502c-4f90-b158-b67f88934c25 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:28.441227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96480370-d2c4-47b4-81ac-96ce42d6b054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:28.442066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96480370-d2c4-47b4-81ac-96ce42d6b054 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:28.442372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96480370-d2c4-47b4-81ac-96ce42d6b054 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:28.442632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96480370-d2c4-47b4-81ac-96ce42d6b054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:28.457293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96480370-d2c4-47b4-81ac-96ce42d6b054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:28.458786 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96480370-d2c4-47b4-81ac-96ce42d6b054 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-96480370-d2c4-47b4-81ac-96ce42d6b054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:28.482863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:28.484603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:28.486280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:28.486280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:28.486280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:28.486280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:28.486280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 367/747] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:28.486280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96480370-d2c4-47b4-81ac-96ce42d6b054 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:28.518827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:28.564665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6859f01-502c-4f90-b158-b67f88934c25 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:28.624203 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586e1cb-dc2b-41d6-bfd3-09174b52d8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:28.625471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586e1cb-dc2b-41d6-bfd3-09174b52d8ee None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:28.625842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586e1cb-dc2b-41d6-bfd3-09174b52d8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:28.626290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586e1cb-dc2b-41d6-bfd3-09174b52d8ee None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:28.627537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586e1cb-dc2b-41d6-bfd3-09174b52d8ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:28.629978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586e1cb-dc2b-41d6-bfd3-09174b52d8ee None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:28.653735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 381/748] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:20:28 2022] POST /identity/v3/services => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:28.655133 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586e1cb-dc2b-41d6-bfd3-09174b52d8ee None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:28.663449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:28.676470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96480370-d2c4-47b4-81ac-96ce42d6b054 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:28.754155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816a4d0-0485-468a-91c3-0db936050350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:28.755654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816a4d0-0485-468a-91c3-0db936050350 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:28.757039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816a4d0-0485-468a-91c3-0db936050350 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:28.757039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816a4d0-0485-468a-91c3-0db936050350 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:28.757793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816a4d0-0485-468a-91c3-0db936050350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:28.758105 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816a4d0-0485-468a-91c3-0db936050350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:28.759549 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816a4d0-0485-468a-91c3-0db936050350 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:28.774509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 368/749] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:28.775390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816a4d0-0485-468a-91c3-0db936050350 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:28.821830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586e1cb-dc2b-41d6-bfd3-09174b52d8ee None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:28.873949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64758dc6-aab5-49c0-892e-bc0cdb6dcc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:28.875166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64758dc6-aab5-49c0-892e-bc0cdb6dcc45 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:28.875716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64758dc6-aab5-49c0-892e-bc0cdb6dcc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:28.876194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64758dc6-aab5-49c0-892e-bc0cdb6dcc45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:28.887591 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:28.893730 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64758dc6-aab5-49c0-892e-bc0cdb6dcc45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:28.896345 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64758dc6-aab5-49c0-892e-bc0cdb6dcc45 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:28.920755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816a4d0-0485-468a-91c3-0db936050350 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-64758dc6-aab5-49c0-892e-bc0cdb6dcc45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:28.929634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:28.930604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:28.931519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:28.931519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:28.931519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:28.931519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:28.932178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 382/750] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:28.933520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64758dc6-aab5-49c0-892e-bc0cdb6dcc45 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:28.977193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a95dc0-cd8d-49f9-9c67-f41e329b2cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:28.978289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a95dc0-cd8d-49f9-9c67-f41e329b2cfe None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:28.978775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a95dc0-cd8d-49f9-9c67-f41e329b2cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:28.979284 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a95dc0-cd8d-49f9-9c67-f41e329b2cfe None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:28.982031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a95dc0-cd8d-49f9-9c67-f41e329b2cfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:28.982581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a95dc0-cd8d-49f9-9c67-f41e329b2cfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:28.985282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a95dc0-cd8d-49f9-9c67-f41e329b2cfe None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:29.008218 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 369/751] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:29.010349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a95dc0-cd8d-49f9-9c67-f41e329b2cfe None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:29.038772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:29.077931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64758dc6-aab5-49c0-892e-bc0cdb6dcc45 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:29.123445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a993075-eaa5-48d6-8eef-c9fbe0dd59b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:29.125037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a993075-eaa5-48d6-8eef-c9fbe0dd59b9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.125806 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a993075-eaa5-48d6-8eef-c9fbe0dd59b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.126258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a993075-eaa5-48d6-8eef-c9fbe0dd59b9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.136158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a993075-eaa5-48d6-8eef-c9fbe0dd59b9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:29.138258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a993075-eaa5-48d6-8eef-c9fbe0dd59b9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-0a993075-eaa5-48d6-8eef-c9fbe0dd59b9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:29.159194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:29.160247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:29.161241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:29.161241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:29.161241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:29.161241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:29.161241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:29.161241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:29.161241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:29.161241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:29.161241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:29.161241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:29.165056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 383/752] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:29.166070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a993075-eaa5-48d6-8eef-c9fbe0dd59b9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:29.219337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a95dc0-cd8d-49f9-9c67-f41e329b2cfe None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:29.254278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:29.275996 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd408d2d-0e96-4a25-a42f-f070552060a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:29.278287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd408d2d-0e96-4a25-a42f-f070552060a8 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.278942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd408d2d-0e96-4a25-a42f-f070552060a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.279445 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd408d2d-0e96-4a25-a42f-f070552060a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.280872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd408d2d-0e96-4a25-a42f-f070552060a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:29.281595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd408d2d-0e96-4a25-a42f-f070552060a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:29.284415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd408d2d-0e96-4a25-a42f-f070552060a8 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:29.304570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 370/753] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:29.305574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd408d2d-0e96-4a25-a42f-f070552060a8 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:29.332448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a993075-eaa5-48d6-8eef-c9fbe0dd59b9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:29.380538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1fb82-d7d7-454e-b135-b5f35327e8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:29.382302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1fb82-d7d7-454e-b135-b5f35327e8bd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.383164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1fb82-d7d7-454e-b135-b5f35327e8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.383792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1fb82-d7d7-454e-b135-b5f35327e8bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.394199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1fb82-d7d7-454e-b135-b5f35327e8bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:29.396884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1fb82-d7d7-454e-b135-b5f35327e8bd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:29.400400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-3eb1fb82-d7d7-454e-b135-b5f35327e8bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:29.415804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:29.416820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:29.418342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:29.418342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:29.418342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:29.418342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:29.418342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:29.418342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:29.418342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:29.418342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:29.421256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 384/754] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:29.422677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1fb82-d7d7-454e-b135-b5f35327e8bd None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:29.447584 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd408d2d-0e96-4a25-a42f-f070552060a8 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:29.500333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b69de6-0536-4212-b8c3-d37c837a388a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:29.502118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b69de6-0536-4212-b8c3-d37c837a388a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.502634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b69de6-0536-4212-b8c3-d37c837a388a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.503337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b69de6-0536-4212-b8c3-d37c837a388a None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.504423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b69de6-0536-4212-b8c3-d37c837a388a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:29.505219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b69de6-0536-4212-b8c3-d37c837a388a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:29.507404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b69de6-0536-4212-b8c3-d37c837a388a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:29.533564 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 371/755] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jul 21 22:20:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:29.535474 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b69de6-0536-4212-b8c3-d37c837a388a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:29.578848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1fb82-d7d7-454e-b135-b5f35327e8bd None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:29.596009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:29.640066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713da39a-6f51-431a-bb0d-7f0753aed6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:29.641357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713da39a-6f51-431a-bb0d-7f0753aed6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.641357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713da39a-6f51-431a-bb0d-7f0753aed6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.641664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713da39a-6f51-431a-bb0d-7f0753aed6f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.649902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713da39a-6f51-431a-bb0d-7f0753aed6f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:29.652007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713da39a-6f51-431a-bb0d-7f0753aed6f2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-713da39a-6f51-431a-bb0d-7f0753aed6f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:29.671072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:29.672382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:29.673397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:29.673397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:29.673397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:29.673397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:29.673397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:29.673397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:29.673397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:29.673397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:29.673397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 385/756] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:29.673897 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713da39a-6f51-431a-bb0d-7f0753aed6f2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:29.749296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b69de6-0536-4212-b8c3-d37c837a388a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:29.798745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:29.808255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e8b08-1ae5-4a2d-86da-38eb8d1402f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:29.809291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e8b08-1ae5-4a2d-86da-38eb8d1402f5 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.809752 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e8b08-1ae5-4a2d-86da-38eb8d1402f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.810178 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e8b08-1ae5-4a2d-86da-38eb8d1402f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.811814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e8b08-1ae5-4a2d-86da-38eb8d1402f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:29.812345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e8b08-1ae5-4a2d-86da-38eb8d1402f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:29.814410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e8b08-1ae5-4a2d-86da-38eb8d1402f5 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:29.832668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 372/757] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:29.835140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e8b08-1ae5-4a2d-86da-38eb8d1402f5 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:29.853333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713da39a-6f51-431a-bb0d-7f0753aed6f2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:29.913253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53404d-ed64-4baf-93d6-58d32351b316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:29.914690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53404d-ed64-4baf-93d6-58d32351b316 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:29.915520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53404d-ed64-4baf-93d6-58d32351b316 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:29.915984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53404d-ed64-4baf-93d6-58d32351b316 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:29.972029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:30.015045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e8b08-1ae5-4a2d-86da-38eb8d1402f5 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:30.022407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53404d-ed64-4baf-93d6-58d32351b316 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:30.023927 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53404d-ed64-4baf-93d6-58d32351b316 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:30.065793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5bf6d-7d8f-4019-a188-80fb4319f5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:30.066833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5bf6d-7d8f-4019-a188-80fb4319f5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:30.067123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5bf6d-7d8f-4019-a188-80fb4319f5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:30.067422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5bf6d-7d8f-4019-a188-80fb4319f5e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:30.068730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5bf6d-7d8f-4019-a188-80fb4319f5e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:30.069291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5bf6d-7d8f-4019-a188-80fb4319f5e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:30.072042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5bf6d-7d8f-4019-a188-80fb4319f5e7 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:30.100976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 373/758] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:30.102248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5bf6d-7d8f-4019-a188-80fb4319f5e7 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:30.161088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 386/759] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:20:30.165696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53404d-ed64-4baf-93d6-58d32351b316 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:30.193491 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:30.255913 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:30.287023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5bf6d-7d8f-4019-a188-80fb4319f5e7 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:30.303317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53404d-ed64-4baf-93d6-58d32351b316 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:30.342585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37c8b7-70fa-46a1-b486-c5de4191f253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:30.343590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37c8b7-70fa-46a1-b486-c5de4191f253 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:30.344346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37c8b7-70fa-46a1-b486-c5de4191f253 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:30.344815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37c8b7-70fa-46a1-b486-c5de4191f253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:30.346734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37c8b7-70fa-46a1-b486-c5de4191f253 None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=d53f6748398d45358f7b2d75ec273dd6) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:30.347352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37c8b7-70fa-46a1-b486-c5de4191f253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:30.349115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37c8b7-70fa-46a1-b486-c5de4191f253 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:30.352465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dd0f8-e364-4fd1-a785-68d2fe1ae043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:30.354081 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dd0f8-e364-4fd1-a785-68d2fe1ae043 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:30.355125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dd0f8-e364-4fd1-a785-68d2fe1ae043 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:30.355125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dd0f8-e364-4fd1-a785-68d2fe1ae043 None admin] PATH_INFO: `/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/c77f6ccc7cb84c04ac532007a04a0f5e/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:30.358338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 374/760] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:30 2022] GET /identity/v3/role_assignments?user.id=d53f6748398d45358f7b2d75ec273dd6&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:30.358644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dd0f8-e364-4fd1-a785-68d2fe1ae043 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0638fe2ce7a40299f2d654f58ed535c, user_id=c77f6ccc7cb84c04ac532007a04a0f5e, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:30.359563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37c8b7-70fa-46a1-b486-c5de4191f253 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:30.367803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dd0f8-e364-4fd1-a785-68d2fe1ae043 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:30.406418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 387/761] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:30 2022] PUT /identity/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/c77f6ccc7cb84c04ac532007a04a0f5e/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:20:30.498160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37c8b7-70fa-46a1-b486-c5de4191f253 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:30.516883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:30.547722 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333cdbc-aa14-4727-ac71-5057a23788ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:30.548856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333cdbc-aa14-4727-ac71-5057a23788ac None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:30.549296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333cdbc-aa14-4727-ac71-5057a23788ac None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:30.549748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333cdbc-aa14-4727-ac71-5057a23788ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:30.574013 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333cdbc-aa14-4727-ac71-5057a23788ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:30.575794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333cdbc-aa14-4727-ac71-5057a23788ac None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-e333cdbc-aa14-4727-ac71-5057a23788ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:30.600006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:30.601226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:30.602345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:30.602345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:30.602345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:30.602345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:30.608910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 375/762] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:30.612421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dd0f8-e364-4fd1-a785-68d2fe1ae043 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:30.781110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dd0f8-e364-4fd1-a785-68d2fe1ae043 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:30.817423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716c891-a23a-4f39-939d-54a693bc0181 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:30.817940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716c891-a23a-4f39-939d-54a693bc0181 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:30.818448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716c891-a23a-4f39-939d-54a693bc0181 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:30.819122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:30.821307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 376/763] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:30.831097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb01266-5b1a-4489-93ba-c8288c9e6f6d None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:30.831607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb01266-5b1a-4489-93ba-c8288c9e6f6d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:30.832105 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb01266-5b1a-4489-93ba-c8288c9e6f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:30.834136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36f14f-7570-4438-bc43-37c0c7f93b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:30.835054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36f14f-7570-4438-bc43-37c0c7f93b20 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:30.835593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36f14f-7570-4438-bc43-37c0c7f93b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:30.836132 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36f14f-7570-4438-bc43-37c0c7f93b20 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:30.837144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36f14f-7570-4438-bc43-37c0c7f93b20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:30.837635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36f14f-7570-4438-bc43-37c0c7f93b20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:30.840290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36f14f-7570-4438-bc43-37c0c7f93b20 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:30.853616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 388/764] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:30.865600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36f14f-7570-4438-bc43-37c0c7f93b20 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:30.873692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-aeb01266-5b1a-4489-93ba-c8288c9e6f6d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:30.879715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-aeb01266-5b1a-4489-93ba-c8288c9e6f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:31.014321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 377/765] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:31.059765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36f14f-7570-4438-bc43-37c0c7f93b20 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:31.085115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb01266-5b1a-4489-93ba-c8288c9e6f6d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:31.123465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b4ccb-6f9f-438a-a2b0-3f952d5d1898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:31.124575 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b4ccb-6f9f-438a-a2b0-3f952d5d1898 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.125012 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b4ccb-6f9f-438a-a2b0-3f952d5d1898 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.125435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b4ccb-6f9f-438a-a2b0-3f952d5d1898 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.133511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b4ccb-6f9f-438a-a2b0-3f952d5d1898 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:31.135373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b4ccb-6f9f-438a-a2b0-3f952d5d1898 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-266b4ccb-6f9f-438a-a2b0-3f952d5d1898 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:31.158273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:31.159299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:31.160318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:31.160318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:31.160318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:31.160318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:31.160318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:31.160318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:31.160318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:31.160318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:31.162911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 389/766] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:31.164636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290a36a-1654-44c8-abfa-871a3bebd29e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.165077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290a36a-1654-44c8-abfa-871a3bebd29e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.165501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290a36a-1654-44c8-abfa-871a3bebd29e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.167514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 390/767] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:31.170501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290a36a-1654-44c8-abfa-871a3bebd29e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:31.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:31.250570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:31.261452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb01266-5b1a-4489-93ba-c8288c9e6f6d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:31.309298 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01435b5c-8c35-4190-b6cb-56a0af6658a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:31.310791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01435b5c-8c35-4190-b6cb-56a0af6658a0 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.311437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01435b5c-8c35-4190-b6cb-56a0af6658a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.312026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01435b5c-8c35-4190-b6cb-56a0af6658a0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.312957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01435b5c-8c35-4190-b6cb-56a0af6658a0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:31.315337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01435b5c-8c35-4190-b6cb-56a0af6658a0 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:31.318722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290a36a-1654-44c8-abfa-871a3bebd29e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-01435b5c-8c35-4190-b6cb-56a0af6658a0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:31.332694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:31.334065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:20:31.335100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:31.339310 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 378/768] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:31 2022] GET /identity/v3/services/network => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:31.341278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08049e-0d35-4f1a-92df-5856052d144c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.341919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08049e-0d35-4f1a-92df-5856052d144c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.342383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08049e-0d35-4f1a-92df-5856052d144c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.369256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-3c08049e-0d35-4f1a-92df-5856052d144c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:31.372351 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3c08049e-0d35-4f1a-92df-5856052d144c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:31.381955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcda719-916a-4ded-ae6f-0267e57088a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:31.383109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcda719-916a-4ded-ae6f-0267e57088a2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.383616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcda719-916a-4ded-ae6f-0267e57088a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.384116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcda719-916a-4ded-ae6f-0267e57088a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.386339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcda719-916a-4ded-ae6f-0267e57088a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:31.386929 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcda719-916a-4ded-ae6f-0267e57088a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:31.389784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcda719-916a-4ded-ae6f-0267e57088a2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:31.433193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 391/769] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:31.437002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcda719-916a-4ded-ae6f-0267e57088a2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:31.469388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 379/770] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:31.470447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08049e-0d35-4f1a-92df-5856052d144c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:31.607458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:31.614297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08049e-0d35-4f1a-92df-5856052d144c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:31.632966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcda719-916a-4ded-ae6f-0267e57088a2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:31.652266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:31.657065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf9885-9d11-4d30-b276-b19f3605fb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:31.658439 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf9885-9d11-4d30-b276-b19f3605fb0f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.658732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf9885-9d11-4d30-b276-b19f3605fb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.659012 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf9885-9d11-4d30-b276-b19f3605fb0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.659935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf9885-9d11-4d30-b276-b19f3605fb0f None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=8fec30ef0b8c41dca643cfefebfe904b) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:31.660258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf9885-9d11-4d30-b276-b19f3605fb0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:31.661785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf9885-9d11-4d30-b276-b19f3605fb0f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:31.674918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 380/771] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:31 2022] GET /identity/v3/role_assignments?user.id=8fec30ef0b8c41dca643cfefebfe904b&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=f957e09d047b47cfa0891d25b4181263 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:31.674918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf9885-9d11-4d30-b276-b19f3605fb0f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:31.773285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6de910-5b3e-4260-81ab-342539681958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:31.773983 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de910-5b3e-4260-81ab-342539681958 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.774271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de910-5b3e-4260-81ab-342539681958 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.774550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de910-5b3e-4260-81ab-342539681958 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.775276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6de910-5b3e-4260-81ab-342539681958 None admin] RBAC: Adding query filter params (name=network) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:31.775598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6de910-5b3e-4260-81ab-342539681958 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:31.778195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6de910-5b3e-4260-81ab-342539681958 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:31.790481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf9885-9d11-4d30-b276-b19f3605fb0f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:31.802749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 392/772] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:31.813595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6de910-5b3e-4260-81ab-342539681958 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:31.831403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d08b8-e5c4-4829-8142-4ae749392895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:31.832236 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d08b8-e5c4-4829-8142-4ae749392895 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:31.832568 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d08b8-e5c4-4829-8142-4ae749392895 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:31.832875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d08b8-e5c4-4829-8142-4ae749392895 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:31.833520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d08b8-e5c4-4829-8142-4ae749392895 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:31.836288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d08b8-e5c4-4829-8142-4ae749392895 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-c99d08b8-e5c4-4829-8142-4ae749392895 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:31.861348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:31.863225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:31.865883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 381/773] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Jul 21 22:20:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:31.875673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d08b8-e5c4-4829-8142-4ae749392895 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:31.920892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:32.093026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6de910-5b3e-4260-81ab-342539681958 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:32.163321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d08b8-e5c4-4829-8142-4ae749392895 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:32.181866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381e834-1862-4332-bca2-e7f2c1ccf9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:32.183688 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381e834-1862-4332-bca2-e7f2c1ccf9ca None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:32.184279 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381e834-1862-4332-bca2-e7f2c1ccf9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:32.184882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381e834-1862-4332-bca2-e7f2c1ccf9ca None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:32.186956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381e834-1862-4332-bca2-e7f2c1ccf9ca None admin] RBAC: Adding query filter params (type=network) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:32.188009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381e834-1862-4332-bca2-e7f2c1ccf9ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:32.190787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381e834-1862-4332-bca2-e7f2c1ccf9ca None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:32.205797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 393/774] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:32.214542 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381e834-1862-4332-bca2-e7f2c1ccf9ca None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:32.267432 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1adc6-68fc-4462-9955-b1d77f1f1e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:32.268597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1adc6-68fc-4462-9955-b1d77f1f1e45 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:32.268597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1adc6-68fc-4462-9955-b1d77f1f1e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:32.268930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1adc6-68fc-4462-9955-b1d77f1f1e45 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:32.269913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1adc6-68fc-4462-9955-b1d77f1f1e45 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:32.270227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1adc6-68fc-4462-9955-b1d77f1f1e45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:32.271771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1adc6-68fc-4462-9955-b1d77f1f1e45 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:32.291575 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 382/775] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jul 21 22:20:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:32.299236 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1adc6-68fc-4462-9955-b1d77f1f1e45 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:32.303689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:32.531075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381e834-1862-4332-bca2-e7f2c1ccf9ca None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:32.591232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:32.650048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1adc6-68fc-4462-9955-b1d77f1f1e45 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:32.655298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282651b-c377-403a-9573-231a1bb40ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:32.655987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282651b-c377-403a-9573-231a1bb40ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:32.656273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282651b-c377-403a-9573-231a1bb40ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:32.656580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282651b-c377-403a-9573-231a1bb40ce1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:32.657405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282651b-c377-403a-9573-231a1bb40ce1 None admin] RBAC: Adding query filter params (interface=public, service_id=8cd0c431728448849a9245408fc4d0ec, region_id=RegionOne) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:32.657722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282651b-c377-403a-9573-231a1bb40ce1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:32.659153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282651b-c377-403a-9573-231a1bb40ce1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:32.678575 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 394/776] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 22:20:32 2022] GET /identity/v3/endpoints?service_id=8cd0c431728448849a9245408fc4d0ec&interface=public®ion_id=RegionOne => generated 194 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:32.687079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282651b-c377-403a-9573-231a1bb40ce1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:32.711069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035ab4f-e574-449e-bde2-bb3bdeb88c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:32.712013 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035ab4f-e574-449e-bde2-bb3bdeb88c41 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:32.712564 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035ab4f-e574-449e-bde2-bb3bdeb88c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:32.713078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035ab4f-e574-449e-bde2-bb3bdeb88c41 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:32.715658 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035ab4f-e574-449e-bde2-bb3bdeb88c41 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:32.716231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035ab4f-e574-449e-bde2-bb3bdeb88c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:32.718835 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035ab4f-e574-449e-bde2-bb3bdeb88c41 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:32.730497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 383/777] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jul 21 22:20:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:32.739584 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035ab4f-e574-449e-bde2-bb3bdeb88c41 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:32.891083 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282651b-c377-403a-9573-231a1bb40ce1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:32.897483 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:32.955785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300dfc5-32be-4e75-b833-5d90ec3fd6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:32.956582 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300dfc5-32be-4e75-b833-5d90ec3fd6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:32.956878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300dfc5-32be-4e75-b833-5d90ec3fd6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:32.957144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300dfc5-32be-4e75-b833-5d90ec3fd6c0 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:32.958259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300dfc5-32be-4e75-b833-5d90ec3fd6c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:32.960409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300dfc5-32be-4e75-b833-5d90ec3fd6c0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:32.969653 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 395/778] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 22:20:32 2022] GET /identity/v3/services => generated 772 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:33.063658 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035ab4f-e574-449e-bde2-bb3bdeb88c41 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:33.127496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6033f-8b80-4184-8861-3a503c946dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:33.128201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6033f-8b80-4184-8861-3a503c946dcf None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.128520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6033f-8b80-4184-8861-3a503c946dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.128801 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6033f-8b80-4184-8861-3a503c946dcf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.129648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6033f-8b80-4184-8861-3a503c946dcf None admin] RBAC: Adding query filter params (interface=public, service_id=4ac585b9f5264bb29b97d0dd107d62e9, region_id=RegionOne) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:33.129972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6033f-8b80-4184-8861-3a503c946dcf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:33.131422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6033f-8b80-4184-8861-3a503c946dcf None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:33.144242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 384/779] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 22:20:32 2022] GET /identity/v3/endpoints?service_id=4ac585b9f5264bb29b97d0dd107d62e9&interface=public®ion_id=RegionOne => generated 194 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:33.152325 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300dfc5-32be-4e75-b833-5d90ec3fd6c0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:33.282759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300dfc5-32be-4e75-b833-5d90ec3fd6c0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:33.322404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8912fa-d3ec-4d51-a71b-310b9eba247f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:33.323159 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8912fa-d3ec-4d51-a71b-310b9eba247f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.323452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8912fa-d3ec-4d51-a71b-310b9eba247f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.323715 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8912fa-d3ec-4d51-a71b-310b9eba247f None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.324416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8912fa-d3ec-4d51-a71b-310b9eba247f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:33.325858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8912fa-d3ec-4d51-a71b-310b9eba247f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:33.332607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 396/780] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 22:20:33 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:33.535909 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0d164-5afd-414f-84e5-8b70816ba769 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.536329 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0d164-5afd-414f-84e5-8b70816ba769 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.536506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0d164-5afd-414f-84e5-8b70816ba769 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.538244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 385/781] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:33.544466 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815028a0-7901-43e6-9b13-78a32ef78f99 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.545219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815028a0-7901-43e6-9b13-78a32ef78f99 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.545219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815028a0-7901-43e6-9b13-78a32ef78f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.569444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-815028a0-7901-43e6-9b13-78a32ef78f99 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:33.570802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-815028a0-7901-43e6-9b13-78a32ef78f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:33.580419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a0108-153d-46ab-ba73-2fd11cca45a1 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.580836 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a0108-153d-46ab-ba73-2fd11cca45a1 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.581058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a0108-153d-46ab-ba73-2fd11cca45a1 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.582774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 386/782] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:33.589307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c43459-aa22-4fe7-a010-b6707e0601e6 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:33.591597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c43459-aa22-4fe7-a010-b6707e0601e6 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:33.591597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c43459-aa22-4fe7-a010-b6707e0601e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:33.617313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:33.623377 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-27c43459-aa22-4fe7-a010-b6707e0601e6 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:33.625809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-27c43459-aa22-4fe7-a010-b6707e0601e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:33.633889 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:33.695143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 397/783] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:33.723358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 387/784] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:33.740021 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815028a0-7901-43e6-9b13-78a32ef78f99 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:33.756719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c43459-aa22-4fe7-a010-b6707e0601e6 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:33.954793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815028a0-7901-43e6-9b13-78a32ef78f99 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:33.966886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c43459-aa22-4fe7-a010-b6707e0601e6 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:34.040764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e600f9-eab6-4b48-b3f2-749f215f6a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:34.042748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e600f9-eab6-4b48-b3f2-749f215f6a26 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:34.043693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e600f9-eab6-4b48-b3f2-749f215f6a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:34.044701 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e600f9-eab6-4b48-b3f2-749f215f6a26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:34.055844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb703597-8483-4e98-a4fd-a3359e33d7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:34.057050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703597-8483-4e98-a4fd-a3359e33d7fb None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:34.058308 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703597-8483-4e98-a4fd-a3359e33d7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:34.059364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb703597-8483-4e98-a4fd-a3359e33d7fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:34.078964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e600f9-eab6-4b48-b3f2-749f215f6a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:34.087490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb703597-8483-4e98-a4fd-a3359e33d7fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-66e600f9-eab6-4b48-b3f2-749f215f6a26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:34.095604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:34.097310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:34.098789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:34.098789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:34.098789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:34.098789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:34.098789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:34.098789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:34.098789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:34.098789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:34.098789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:34.098789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:34.098789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:34.098789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-fb703597-8483-4e98-a4fd-a3359e33d7fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:34.099912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:34.101436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:34.102921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:34.102921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:34.102921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:34.102921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:34.102921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:34.102921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:34.102921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:34.102921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:34.102921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:34.102921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:34.102921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:34.102921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:34.108693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:34.110074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 398/785] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:34.112531 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 388/786] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:34.117578 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e600f9-eab6-4b48-b3f2-749f215f6a26 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:34.118723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb703597-8483-4e98-a4fd-a3359e33d7fb None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:34.153232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:34.295987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e600f9-eab6-4b48-b3f2-749f215f6a26 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:34.299169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb703597-8483-4e98-a4fd-a3359e33d7fb None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:34.335395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cf404-7a11-4c76-b0f9-32115426b77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:34.336588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cf404-7a11-4c76-b0f9-32115426b77c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:34.337344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cf404-7a11-4c76-b0f9-32115426b77c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:34.338054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cf404-7a11-4c76-b0f9-32115426b77c None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:34.339121 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638cf404-7a11-4c76-b0f9-32115426b77c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:34.339589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638cf404-7a11-4c76-b0f9-32115426b77c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:34.341429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638cf404-7a11-4c76-b0f9-32115426b77c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:34.342195 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e6c38-87c5-4ad0-ab57-5583cc1f7cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:34.342947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e6c38-87c5-4ad0-ab57-5583cc1f7cde None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:34.343571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e6c38-87c5-4ad0-ab57-5583cc1f7cde None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:34.343934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e6c38-87c5-4ad0-ab57-5583cc1f7cde None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:34.345259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e6c38-87c5-4ad0-ab57-5583cc1f7cde None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:34.345647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e6c38-87c5-4ad0-ab57-5583cc1f7cde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:34.347086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e6c38-87c5-4ad0-ab57-5583cc1f7cde None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:34.353040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 399/787] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:34.355244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec39c0-d2db-4a1b-ba24-827c44fb585e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:34.355679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec39c0-d2db-4a1b-ba24-827c44fb585e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:34.356093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec39c0-d2db-4a1b-ba24-827c44fb585e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:34.357875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 400/788] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:34.359729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec39c0-d2db-4a1b-ba24-827c44fb585e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:34.368519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 389/789] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:34.368519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1a4b9-4cd1-4c5c-91a4-73a28595f589 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:34.368519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1a4b9-4cd1-4c5c-91a4-73a28595f589 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:34.368519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1a4b9-4cd1-4c5c-91a4-73a28595f589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:34.390878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-50b1a4b9-4cd1-4c5c-91a4-73a28595f589 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:34.392527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-50b1a4b9-4cd1-4c5c-91a4-73a28595f589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:34.481565 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:34.485281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:34.499038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 390/790] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:34.500719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1a4b9-4cd1-4c5c-91a4-73a28595f589 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:34.512213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec39c0-d2db-4a1b-ba24-827c44fb585e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:34.554324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773d369-a139-4b45-994a-1d21fe249e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:34.555128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773d369-a139-4b45-994a-1d21fe249e80 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:34.555290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773d369-a139-4b45-994a-1d21fe249e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:34.555614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773d369-a139-4b45-994a-1d21fe249e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:34.569246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773d369-a139-4b45-994a-1d21fe249e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:34.570826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773d369-a139-4b45-994a-1d21fe249e80 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b773d369-a139-4b45-994a-1d21fe249e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:34.592046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:34.593232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:34.594205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:34.594205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:34.594205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:34.594205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:34.598261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 401/791] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:34.598953 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773d369-a139-4b45-994a-1d21fe249e80 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:34.657712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1a4b9-4cd1-4c5c-91a4-73a28595f589 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:34.705442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2b6c3-60c4-4699-85c9-10515dbfe4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:34.706312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2b6c3-60c4-4699-85c9-10515dbfe4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:34.706599 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2b6c3-60c4-4699-85c9-10515dbfe4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:34.706878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2b6c3-60c4-4699-85c9-10515dbfe4a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:34.724211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2b6c3-60c4-4699-85c9-10515dbfe4a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:34.726621 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2b6c3-60c4-4699-85c9-10515dbfe4a3 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:34.727760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773d369-a139-4b45-994a-1d21fe249e80 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-c5e2b6c3-60c4-4699-85c9-10515dbfe4a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:34.747552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:34.748550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:34.749440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:34.749440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:34.749440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:34.749440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:34.751815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 391/792] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:34.752891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2b6c3-60c4-4699-85c9-10515dbfe4a3 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:34.766880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:34.773988 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d999037-9477-430c-a595-ce0424501981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:34.775247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d999037-9477-430c-a595-ce0424501981 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:34.775536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d999037-9477-430c-a595-ce0424501981 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:34.775819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d999037-9477-430c-a595-ce0424501981 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:34.787877 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d999037-9477-430c-a595-ce0424501981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-9d999037-9477-430c-a595-ce0424501981 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:34.796392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:34.797428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:34.798286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 402/793] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:34.798994 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d999037-9477-430c-a595-ce0424501981 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:34.829903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:34.906171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2b6c3-60c4-4699-85c9-10515dbfe4a3 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:34.912445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d999037-9477-430c-a595-ce0424501981 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:34.948178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e1eb7-e726-421b-9a82-42b5a969eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:34.948894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e1eb7-e726-421b-9a82-42b5a969eea4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:34.949182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e1eb7-e726-421b-9a82-42b5a969eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:34.949459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e1eb7-e726-421b-9a82-42b5a969eea4 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:34.950188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e1eb7-e726-421b-9a82-42b5a969eea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:34.950499 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e1eb7-e726-421b-9a82-42b5a969eea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:34.952003 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e1eb7-e726-421b-9a82-42b5a969eea4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:34.959690 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa8692-e4ff-4f85-8c93-9c20629853e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:34.960737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa8692-e4ff-4f85-8c93-9c20629853e0 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:34.961358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa8692-e4ff-4f85-8c93-9c20629853e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:34.961711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa8692-e4ff-4f85-8c93-9c20629853e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:34.962556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa8692-e4ff-4f85-8c93-9c20629853e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:34.962914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa8692-e4ff-4f85-8c93-9c20629853e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:34.964927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa8692-e4ff-4f85-8c93-9c20629853e0 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:34.970565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 403/794] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:34.971684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e1eb7-e726-421b-9a82-42b5a969eea4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:34.979232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 392/795] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:34.979232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa8692-e4ff-4f85-8c93-9c20629853e0 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:35.032260 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:35.082842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e1eb7-e726-421b-9a82-42b5a969eea4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:35.089439 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa8692-e4ff-4f85-8c93-9c20629853e0 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:35.092878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:35.121086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1d0f2-f15a-4830-92f3-4515a76ec693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:35.121972 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1d0f2-f15a-4830-92f3-4515a76ec693 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:35.122114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1d0f2-f15a-4830-92f3-4515a76ec693 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:35.122301 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1d0f2-f15a-4830-92f3-4515a76ec693 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:35.123138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1d0f2-f15a-4830-92f3-4515a76ec693 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:35.123443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1d0f2-f15a-4830-92f3-4515a76ec693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:35.124934 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1d0f2-f15a-4830-92f3-4515a76ec693 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:35.129871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a200b8b-ce9b-4877-a539-9bdcca1fc11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:35.130815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a200b8b-ce9b-4877-a539-9bdcca1fc11c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:35.131114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a200b8b-ce9b-4877-a539-9bdcca1fc11c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:35.131402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a200b8b-ce9b-4877-a539-9bdcca1fc11c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:35.134497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 404/796] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:35.136127 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1d0f2-f15a-4830-92f3-4515a76ec693 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:35.145757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a200b8b-ce9b-4877-a539-9bdcca1fc11c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:35.147628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a200b8b-ce9b-4877-a539-9bdcca1fc11c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-5a200b8b-ce9b-4877-a539-9bdcca1fc11c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:35.165361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:35.166420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:35.167966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:35.167966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:35.167966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:35.167966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:35.167966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:35.167966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:35.167966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:35.167966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:35.167966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:35.167966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:35.167966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 393/797] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 22:20:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:35.168747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a200b8b-ce9b-4877-a539-9bdcca1fc11c None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:35.247469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1d0f2-f15a-4830-92f3-4515a76ec693 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:35.261977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:35.285516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a65f10-e891-4097-ad8d-de4716332b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:35.286314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a65f10-e891-4097-ad8d-de4716332b8d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:35.286649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a65f10-e891-4097-ad8d-de4716332b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:35.286929 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a65f10-e891-4097-ad8d-de4716332b8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:35.287690 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a200b8b-ce9b-4877-a539-9bdcca1fc11c None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:35.296015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a65f10-e891-4097-ad8d-de4716332b8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:35.297709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a65f10-e891-4097-ad8d-de4716332b8d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-23a65f10-e891-4097-ad8d-de4716332b8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:35.312968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:35.314213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:35.315400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:35.315400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:35.315400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:35.315400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:35.315400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:35.315400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:35.315400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:35.315400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:35.315400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:20:35.315400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:35.319744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 405/798] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 22:20:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:35.321268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a65f10-e891-4097-ad8d-de4716332b8d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:35.330009 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbcc61-dfba-419d-93dd-872a49a5808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:35.331090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbcc61-dfba-419d-93dd-872a49a5808e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:35.331558 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbcc61-dfba-419d-93dd-872a49a5808e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:35.332109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbcc61-dfba-419d-93dd-872a49a5808e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:35.344408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbcc61-dfba-419d-93dd-872a49a5808e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:35.346329 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbcc61-dfba-419d-93dd-872a49a5808e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-66bbcc61-dfba-419d-93dd-872a49a5808e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:35.366165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:35.367346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:35.368402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:35.368402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:35.368402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:35.368402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:35.375028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 394/799] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:35.376181 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbcc61-dfba-419d-93dd-872a49a5808e None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:35.404901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:35.443114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a65f10-e891-4097-ad8d-de4716332b8d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:35.479723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f5823-9bc7-4489-b3a0-def744a42c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:35.480416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f5823-9bc7-4489-b3a0-def744a42c2d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:35.480746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f5823-9bc7-4489-b3a0-def744a42c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:35.481020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f5823-9bc7-4489-b3a0-def744a42c2d None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:35.481867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f5823-9bc7-4489-b3a0-def744a42c2d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:35.482175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f5823-9bc7-4489-b3a0-def744a42c2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:35.483621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f5823-9bc7-4489-b3a0-def744a42c2d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:35.486753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbcc61-dfba-419d-93dd-872a49a5808e None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:35.503238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 406/800] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jul 21 22:20:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:35.503870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f5823-9bc7-4489-b3a0-def744a42c2d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:35.524695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bebf6-7312-44e9-8499-acd95e3079a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:35.525382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bebf6-7312-44e9-8499-acd95e3079a6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:35.525681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bebf6-7312-44e9-8499-acd95e3079a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:35.526472 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bebf6-7312-44e9-8499-acd95e3079a6 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:35.527460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bebf6-7312-44e9-8499-acd95e3079a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:35.527736 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bebf6-7312-44e9-8499-acd95e3079a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:35.529199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bebf6-7312-44e9-8499-acd95e3079a6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:35.536755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:35.555842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 395/801] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 22:20:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:35.556962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bebf6-7312-44e9-8499-acd95e3079a6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:35.614644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f5823-9bc7-4489-b3a0-def744a42c2d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:35.644294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:35.652125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2cb40-9b11-4f6f-b1f5-00aeb154a2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:35.652923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2cb40-9b11-4f6f-b1f5-00aeb154a2de None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:35.653192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2cb40-9b11-4f6f-b1f5-00aeb154a2de None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:35.653484 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2cb40-9b11-4f6f-b1f5-00aeb154a2de None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:35.654186 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2cb40-9b11-4f6f-b1f5-00aeb154a2de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:35.654554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2cb40-9b11-4f6f-b1f5-00aeb154a2de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:35.656035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2cb40-9b11-4f6f-b1f5-00aeb154a2de None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:35.668955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 407/802] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:35.669872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2cb40-9b11-4f6f-b1f5-00aeb154a2de None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:35.674174 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bebf6-7312-44e9-8499-acd95e3079a6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:35.713257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5b6fb-a4bf-4d53-9d9f-8adafa5577ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:35.714608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5b6fb-a4bf-4d53-9d9f-8adafa5577ca None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:35.714901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5b6fb-a4bf-4d53-9d9f-8adafa5577ca None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:35.715166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5b6fb-a4bf-4d53-9d9f-8adafa5577ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:35.726764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5b6fb-a4bf-4d53-9d9f-8adafa5577ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:35.728345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5b6fb-a4bf-4d53-9d9f-8adafa5577ca None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-96f5b6fb-a4bf-4d53-9d9f-8adafa5577ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:35.748517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:35.749518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:35.750499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:35.750499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:35.750499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:35.750499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:35.750499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 396/803] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:35.750843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5b6fb-a4bf-4d53-9d9f-8adafa5577ca None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:35.768725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:35.782800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2cb40-9b11-4f6f-b1f5-00aeb154a2de None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:35.827321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb89fc9-069e-48aa-8790-537a30705db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:35.827983 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb89fc9-069e-48aa-8790-537a30705db6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:35.828266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb89fc9-069e-48aa-8790-537a30705db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:35.828572 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb89fc9-069e-48aa-8790-537a30705db6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:35.856138 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5b6fb-a4bf-4d53-9d9f-8adafa5577ca None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:35.892308 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:35.904529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a28f4-1e28-43cd-bfe0-b5a792117077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:35.905335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a28f4-1e28-43cd-bfe0-b5a792117077 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:35.905622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a28f4-1e28-43cd-bfe0-b5a792117077 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:35.905914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a28f4-1e28-43cd-bfe0-b5a792117077 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:35.908578 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb89fc9-069e-48aa-8790-537a30705db6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:35.910725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb89fc9-069e-48aa-8790-537a30705db6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:35.913556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a28f4-1e28-43cd-bfe0-b5a792117077 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:35.914994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a28f4-1e28-43cd-bfe0-b5a792117077 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-1d9a28f4-1e28-43cd-bfe0-b5a792117077 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:35.930851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:35.931847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:35.933482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:35.933482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:35.933482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:35.933482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:35.933482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:35.933482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:35.933482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:35.933482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:35.933482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:35.933482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:35.933482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 397/804] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:35.934395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591bd2a-9b9b-4d2d-9cd6-6bfa1278c789 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:35.934395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591bd2a-9b9b-4d2d-9cd6-6bfa1278c789 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:35.934686 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591bd2a-9b9b-4d2d-9cd6-6bfa1278c789 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:35.939669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 398/805] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:35.939669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591bd2a-9b9b-4d2d-9cd6-6bfa1278c789 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.016268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:36.029509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 408/806] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:20:36.030896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb89fc9-069e-48aa-8790-537a30705db6 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.055603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591bd2a-9b9b-4d2d-9cd6-6bfa1278c789 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:36.094767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69792be6-d520-497e-b39a-7479c7a1c4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:36.095509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69792be6-d520-497e-b39a-7479c7a1c4de None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.095904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69792be6-d520-497e-b39a-7479c7a1c4de None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.096184 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69792be6-d520-497e-b39a-7479c7a1c4de None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.096956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69792be6-d520-497e-b39a-7479c7a1c4de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:36.097263 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69792be6-d520-497e-b39a-7479c7a1c4de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:36.098774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69792be6-d520-497e-b39a-7479c7a1c4de None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:36.116415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 399/807] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:36.116415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bff08d-9193-4c2e-b30e-7e1618fa95f9 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.116415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bff08d-9193-4c2e-b30e-7e1618fa95f9 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.116415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bff08d-9193-4c2e-b30e-7e1618fa95f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.131713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-97bff08d-9193-4c2e-b30e-7e1618fa95f9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:36.132943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-97bff08d-9193-4c2e-b30e-7e1618fa95f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:36.144901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb89fc9-069e-48aa-8790-537a30705db6 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:36.180827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121e12a-7fa7-4d85-961e-36c9b7cc566b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:36.181458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:36.181881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121e12a-7fa7-4d85-961e-36c9b7cc566b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.182456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121e12a-7fa7-4d85-961e-36c9b7cc566b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.182736 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121e12a-7fa7-4d85-961e-36c9b7cc566b None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.183832 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121e12a-7fa7-4d85-961e-36c9b7cc566b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:36.184146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121e12a-7fa7-4d85-961e-36c9b7cc566b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:36.185756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121e12a-7fa7-4d85-961e-36c9b7cc566b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:36.204031 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 409/808] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:36.205207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121e12a-7fa7-4d85-961e-36c9b7cc566b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.214509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 400/809] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:36.215753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bff08d-9193-4c2e-b30e-7e1618fa95f9 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.274171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:36.337293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121e12a-7fa7-4d85-961e-36c9b7cc566b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:36.345667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bff08d-9193-4c2e-b30e-7e1618fa95f9 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:36.385109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc787c1-9d25-4fb4-b997-2e18d96baf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:36.387191 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc787c1-9d25-4fb4-b997-2e18d96baf86 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.387695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc787c1-9d25-4fb4-b997-2e18d96baf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.388205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc787c1-9d25-4fb4-b997-2e18d96baf86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.390882 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1eb2ed-4ed6-43dc-8fa4-9adde95eada8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:36.391702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1eb2ed-4ed6-43dc-8fa4-9adde95eada8 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.391981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1eb2ed-4ed6-43dc-8fa4-9adde95eada8 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.392251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1eb2ed-4ed6-43dc-8fa4-9adde95eada8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.400041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1eb2ed-4ed6-43dc-8fa4-9adde95eada8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:36.401537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1eb2ed-4ed6-43dc-8fa4-9adde95eada8 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:36.408239 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc787c1-9d25-4fb4-b997-2e18d96baf86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:36.411292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc787c1-9d25-4fb4-b997-2e18d96baf86 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-ad1eb2ed-4ed6-43dc-8fa4-9adde95eada8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:36.417282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:36.419025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:36.420892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:36.420892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:36.420892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:36.420892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:36.420892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:36.420892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:36.420892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:36.420892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:36.420892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 401/810] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:36.420892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1eb2ed-4ed6-43dc-8fa4-9adde95eada8 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-abc787c1-9d25-4fb4-b997-2e18d96baf86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:36.439456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:36.440388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:36.441319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:36.441319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:36.441319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:36.441319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:36.441319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 410/811] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:36.441669 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc787c1-9d25-4fb4-b997-2e18d96baf86 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.470598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:36.547394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1eb2ed-4ed6-43dc-8fa4-9adde95eada8 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:36.594101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e21cf9-8802-489b-9fc2-a01904ac2c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:36.594946 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e21cf9-8802-489b-9fc2-a01904ac2c2f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.595231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e21cf9-8802-489b-9fc2-a01904ac2c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.595677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e21cf9-8802-489b-9fc2-a01904ac2c2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.597561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc787c1-9d25-4fb4-b997-2e18d96baf86 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:36.600273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:36.608220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e21cf9-8802-489b-9fc2-a01904ac2c2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:36.609772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e21cf9-8802-489b-9fc2-a01904ac2c2f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-a9e21cf9-8802-489b-9fc2-a01904ac2c2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:36.629403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:36.630791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:36.631707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:36.631707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:36.631707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:36.631707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:36.631707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 402/812] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:36.631707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e21cf9-8802-489b-9fc2-a01904ac2c2f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.671416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c89ef-7779-4578-99c8-a919030a95b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:36.672187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c89ef-7779-4578-99c8-a919030a95b0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.672420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c89ef-7779-4578-99c8-a919030a95b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.672730 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c89ef-7779-4578-99c8-a919030a95b0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.673399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c89ef-7779-4578-99c8-a919030a95b0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:36.674835 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c89ef-7779-4578-99c8-a919030a95b0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-220c89ef-7779-4578-99c8-a919030a95b0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:36.688001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:36.689146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:36.690020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 411/813] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/services/network => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:36.691324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c89ef-7779-4578-99c8-a919030a95b0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.742136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:36.756872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e21cf9-8802-489b-9fc2-a01904ac2c2f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:36.798112 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8696939-6fcf-4e87-b02f-1dae01383b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:36.798819 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696939-6fcf-4e87-b02f-1dae01383b04 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.799120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696939-6fcf-4e87-b02f-1dae01383b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.799382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696939-6fcf-4e87-b02f-1dae01383b04 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.800193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8696939-6fcf-4e87-b02f-1dae01383b04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:36.800547 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8696939-6fcf-4e87-b02f-1dae01383b04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:36.801967 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8696939-6fcf-4e87-b02f-1dae01383b04 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:36.811783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c89ef-7779-4578-99c8-a919030a95b0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:36.815694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 403/814] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:36.817080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851c4ed-2b5b-431f-bb23-e440ffc23324 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.817384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851c4ed-2b5b-431f-bb23-e440ffc23324 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.817630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851c4ed-2b5b-431f-bb23-e440ffc23324 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.824954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 404/815] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:36.824954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851c4ed-2b5b-431f-bb23-e440ffc23324 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.856915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b7b55-b13a-459b-a49a-f1218f98fcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:36.857557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b7b55-b13a-459b-a49a-f1218f98fcb8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:36.857840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b7b55-b13a-459b-a49a-f1218f98fcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:36.858432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b7b55-b13a-459b-a49a-f1218f98fcb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:36.858847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b7b55-b13a-459b-a49a-f1218f98fcb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:36.859164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b7b55-b13a-459b-a49a-f1218f98fcb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:36.860681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b7b55-b13a-459b-a49a-f1218f98fcb8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:36.874961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 412/816] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:36.876130 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b7b55-b13a-459b-a49a-f1218f98fcb8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:36.908550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:36.965737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851c4ed-2b5b-431f-bb23-e440ffc23324 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:36.995493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b7b55-b13a-459b-a49a-f1218f98fcb8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.004789 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3341357-245d-4f44-8a37-0759c536e6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.006273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3341357-245d-4f44-8a37-0759c536e6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.006273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3341357-245d-4f44-8a37-0759c536e6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.006273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3341357-245d-4f44-8a37-0759c536e6f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.006860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3341357-245d-4f44-8a37-0759c536e6f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:37.007083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3341357-245d-4f44-8a37-0759c536e6f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.008552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3341357-245d-4f44-8a37-0759c536e6f8 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.022220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 405/817] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:37.023143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3341357-245d-4f44-8a37-0759c536e6f8 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.035817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.038168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb98ef0-49aa-4f65-9ab7-189211fc296e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.038921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb98ef0-49aa-4f65-9ab7-189211fc296e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.039202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb98ef0-49aa-4f65-9ab7-189211fc296e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.039479 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb98ef0-49aa-4f65-9ab7-189211fc296e None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.040202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb98ef0-49aa-4f65-9ab7-189211fc296e None admin] RBAC: Adding query filter params (name=network) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:37.040535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb98ef0-49aa-4f65-9ab7-189211fc296e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.043264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb98ef0-49aa-4f65-9ab7-189211fc296e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.048241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 413/818] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:37.049576 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ac672-a0a3-4d59-b83f-f543112fce74 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.049859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ac672-a0a3-4d59-b83f-f543112fce74 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.050196 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ac672-a0a3-4d59-b83f-f543112fce74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.074562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ed5ac672-a0a3-4d59-b83f-f543112fce74 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:37.076635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ed5ac672-a0a3-4d59-b83f-f543112fce74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:37.147715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3341357-245d-4f44-8a37-0759c536e6f8 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.168791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 414/819] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:37.169682 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ac672-a0a3-4d59-b83f-f543112fce74 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.198122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.295015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ac672-a0a3-4d59-b83f-f543112fce74 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.314602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaea15d-8177-4dba-bef5-fb4a1350b874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.315498 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaea15d-8177-4dba-bef5-fb4a1350b874 None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.315780 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaea15d-8177-4dba-bef5-fb4a1350b874 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.316054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaea15d-8177-4dba-bef5-fb4a1350b874 None admin] PATH_INFO: `/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/d53f6748398d45358f7b2d75ec273dd6/roles/ca5450234bd44b46a10185c4c8a0647f` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.331777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaea15d-8177-4dba-bef5-fb4a1350b874 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0638fe2ce7a40299f2d654f58ed535c, user_id=d53f6748398d45358f7b2d75ec273dd6, role_id=ca5450234bd44b46a10185c4c8a0647f)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.333259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaea15d-8177-4dba-bef5-fb4a1350b874 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.335386 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65784b-f40c-4869-b747-2963b102683f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.336661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65784b-f40c-4869-b747-2963b102683f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.337176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65784b-f40c-4869-b747-2963b102683f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.337644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65784b-f40c-4869-b747-2963b102683f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.349490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 406/820] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:37 2022] PUT /identity/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/d53f6748398d45358f7b2d75ec273dd6/roles/ca5450234bd44b46a10185c4c8a0647f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:20:37.350981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaea15d-8177-4dba-bef5-fb4a1350b874 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.351713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65784b-f40c-4869-b747-2963b102683f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.354152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65784b-f40c-4869-b747-2963b102683f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-7a65784b-f40c-4869-b747-2963b102683f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:37.382429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:37.383770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:37.384748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:37.384748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:37.384748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:37.384748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:37.384748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:37.384748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:37.384748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:37.384748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:37.385902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 415/821] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:37.387730 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65784b-f40c-4869-b747-2963b102683f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.453369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.506129 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaea15d-8177-4dba-bef5-fb4a1350b874 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.532949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.558083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd200402-c7b9-49b6-8b95-ee1b8d5d2047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.558606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd200402-c7b9-49b6-8b95-ee1b8d5d2047 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.558965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd200402-c7b9-49b6-8b95-ee1b8d5d2047 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.559312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd200402-c7b9-49b6-8b95-ee1b8d5d2047 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.559993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65784b-f40c-4869-b747-2963b102683f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.567651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd200402-c7b9-49b6-8b95-ee1b8d5d2047 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.569155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd200402-c7b9-49b6-8b95-ee1b8d5d2047 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-cd200402-c7b9-49b6-8b95-ee1b8d5d2047 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:37.585661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:37.587441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:37.590435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:37.590435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:37.590435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:37.590435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:37.590435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:37.590435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:37.590435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:37.590435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:37.590435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 407/822] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:37.590435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd200402-c7b9-49b6-8b95-ee1b8d5d2047 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.615168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4e3dd-9b83-41d5-b303-d090c3e71112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.615854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4e3dd-9b83-41d5-b303-d090c3e71112 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.616142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4e3dd-9b83-41d5-b303-d090c3e71112 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.616418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4e3dd-9b83-41d5-b303-d090c3e71112 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.617184 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4e3dd-9b83-41d5-b303-d090c3e71112 None admin] RBAC: Adding query filter params (type=network) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:37.617490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4e3dd-9b83-41d5-b303-d090c3e71112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.618909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4e3dd-9b83-41d5-b303-d090c3e71112 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.625926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 416/823] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:37.626583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4e3dd-9b83-41d5-b303-d090c3e71112 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.732770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd200402-c7b9-49b6-8b95-ee1b8d5d2047 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.754459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.755465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4e3dd-9b83-41d5-b303-d090c3e71112 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.777732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efac3b6-6625-45d3-8490-d020adf1ad2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.778565 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efac3b6-6625-45d3-8490-d020adf1ad2e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.778846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efac3b6-6625-45d3-8490-d020adf1ad2e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.779121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efac3b6-6625-45d3-8490-d020adf1ad2e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.779793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efac3b6-6625-45d3-8490-d020adf1ad2e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.781205 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efac3b6-6625-45d3-8490-d020adf1ad2e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-3efac3b6-6625-45d3-8490-d020adf1ad2e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:37.796651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:37.797861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:20:37.798717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:37.799593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7b0a0-d7e1-4446-a3ba-714237e2ff9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:37.799593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7b0a0-d7e1-4446-a3ba-714237e2ff9c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:37.799593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7b0a0-d7e1-4446-a3ba-714237e2ff9c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:37.799593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7b0a0-d7e1-4446-a3ba-714237e2ff9c None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:37.799879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 408/824] 10.222.0.42 () {56 vars in 1129 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:37.799948 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7b0a0-d7e1-4446-a3ba-714237e2ff9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:37.800016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efac3b6-6625-45d3-8490-d020adf1ad2e None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.800093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7b0a0-d7e1-4446-a3ba-714237e2ff9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:37.806774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7b0a0-d7e1-4446-a3ba-714237e2ff9c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:37.818233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 417/825] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:37.818993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7b0a0-d7e1-4446-a3ba-714237e2ff9c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:37.840792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:37.961778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efac3b6-6625-45d3-8490-d020adf1ad2e None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:37.973967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7b0a0-d7e1-4446-a3ba-714237e2ff9c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:38.003876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd91b15-743b-4f3b-b707-899f0ef14940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:38.004950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd91b15-743b-4f3b-b707-899f0ef14940 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:38.005382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd91b15-743b-4f3b-b707-899f0ef14940 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:38.005801 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd91b15-743b-4f3b-b707-899f0ef14940 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:38.007121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd91b15-743b-4f3b-b707-899f0ef14940 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:38.007601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd91b15-743b-4f3b-b707-899f0ef14940 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:38.010080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd91b15-743b-4f3b-b707-899f0ef14940 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:38.021801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c274d0a-0644-4b04-8c73-19935fb963ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:38.022869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c274d0a-0644-4b04-8c73-19935fb963ca None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:38.023338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c274d0a-0644-4b04-8c73-19935fb963ca None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:38.023769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c274d0a-0644-4b04-8c73-19935fb963ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:38.025147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c274d0a-0644-4b04-8c73-19935fb963ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:38.027572 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c274d0a-0644-4b04-8c73-19935fb963ca None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:38.031949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 409/826] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:38.033195 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd91b15-743b-4f3b-b707-899f0ef14940 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:38.065041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 418/827] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 22:20:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:38.066956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c274d0a-0644-4b04-8c73-19935fb963ca None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:38.155399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:38.189395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:38.255795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd91b15-743b-4f3b-b707-899f0ef14940 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:38.286397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c274d0a-0644-4b04-8c73-19935fb963ca None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:38.315399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abfca6-13e0-4927-adbe-9b3913e1f76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:38.316917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abfca6-13e0-4927-adbe-9b3913e1f76f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:38.317347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abfca6-13e0-4927-adbe-9b3913e1f76f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:38.317854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abfca6-13e0-4927-adbe-9b3913e1f76f None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:38.318758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abfca6-13e0-4927-adbe-9b3913e1f76f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:38.320527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abfca6-13e0-4927-adbe-9b3913e1f76f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:38.320706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abfca6-13e0-4927-adbe-9b3913e1f76f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:38.329437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 410/828] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jul 21 22:20:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:38.330675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abfca6-13e0-4927-adbe-9b3913e1f76f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:38.347898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d641f40-1902-4da4-8c7e-de0c93dd6575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:38.349464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d641f40-1902-4da4-8c7e-de0c93dd6575 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:38.349977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d641f40-1902-4da4-8c7e-de0c93dd6575 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:38.350921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d641f40-1902-4da4-8c7e-de0c93dd6575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:38.352655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d641f40-1902-4da4-8c7e-de0c93dd6575 None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=c77f6ccc7cb84c04ac532007a04a0f5e) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:38.353409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d641f40-1902-4da4-8c7e-de0c93dd6575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:38.358007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d641f40-1902-4da4-8c7e-de0c93dd6575 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:38.370324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 419/829] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:38 2022] GET /identity/v3/role_assignments?user.id=c77f6ccc7cb84c04ac532007a04a0f5e&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=f957e09d047b47cfa0891d25b4181263 => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:38.371836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d641f40-1902-4da4-8c7e-de0c93dd6575 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:38.470724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abfca6-13e0-4927-adbe-9b3913e1f76f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:38.480864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:38.515726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a00232-8dca-473c-bf4b-414a51a098dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:38.516856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a00232-8dca-473c-bf4b-414a51a098dc None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:38.517247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a00232-8dca-473c-bf4b-414a51a098dc None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:38.518462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a00232-8dca-473c-bf4b-414a51a098dc None admin] PATH_INFO: `/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/8fec30ef0b8c41dca643cfefebfe904b/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:38.520288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a00232-8dca-473c-bf4b-414a51a098dc None admin] RBAC: Authorizing `identity:create_grant(project_id=b0638fe2ce7a40299f2d654f58ed535c, user_id=8fec30ef0b8c41dca643cfefebfe904b, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:38.522081 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a00232-8dca-473c-bf4b-414a51a098dc None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:38.545280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 411/830] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:38 2022] PUT /identity/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/8fec30ef0b8c41dca643cfefebfe904b/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:20:38.568935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d641f40-1902-4da4-8c7e-de0c93dd6575 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:38.600054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:38.626455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1699d90-a8c1-4c47-a818-9a45efc1af68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:38.627615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1699d90-a8c1-4c47-a818-9a45efc1af68 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:38.628115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1699d90-a8c1-4c47-a818-9a45efc1af68 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:38.628668 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1699d90-a8c1-4c47-a818-9a45efc1af68 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:38.629668 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1699d90-a8c1-4c47-a818-9a45efc1af68 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:38.630215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1699d90-a8c1-4c47-a818-9a45efc1af68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:38.632132 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1699d90-a8c1-4c47-a818-9a45efc1af68 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:38.651782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 420/831] 10.222.0.42 () {56 vars in 1124 bytes} [Thu Jul 21 22:20:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:38.661058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a00232-8dca-473c-bf4b-414a51a098dc None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:38.798023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a00232-8dca-473c-bf4b-414a51a098dc None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:38.836459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26d904-dcb8-468a-851c-bcff9ff303da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:38.837584 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d904-dcb8-468a-851c-bcff9ff303da None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:38.838074 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d904-dcb8-468a-851c-bcff9ff303da None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:38.838975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d904-dcb8-468a-851c-bcff9ff303da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:38.840177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26d904-dcb8-468a-851c-bcff9ff303da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:38.841870 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26d904-dcb8-468a-851c-bcff9ff303da None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:38.871632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 412/832] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 22:20:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:39.912670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564c720-9cd3-42e3-b2b7-ac831e5a811d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:39.913200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564c720-9cd3-42e3-b2b7-ac831e5a811d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:39.913422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564c720-9cd3-42e3-b2b7-ac831e5a811d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:39.915365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 421/833] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:39.921843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400431-b197-4f9a-b97d-a1154c79cd93 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:39.922338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400431-b197-4f9a-b97d-a1154c79cd93 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:39.922433 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400431-b197-4f9a-b97d-a1154c79cd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:39.942497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-a5400431-b197-4f9a-b97d-a1154c79cd93 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:39.943868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a5400431-b197-4f9a-b97d-a1154c79cd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:39.963909 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:40.024973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 413/834] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:40.048915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564c720-9cd3-42e3-b2b7-ac831e5a811d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:40.170254 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564c720-9cd3-42e3-b2b7-ac831e5a811d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:40.208448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a1d5d-2264-4c4f-8a2c-ac5b06d1e741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:40.209381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a1d5d-2264-4c4f-8a2c-ac5b06d1e741 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.209666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a1d5d-2264-4c4f-8a2c-ac5b06d1e741 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.209940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a1d5d-2264-4c4f-8a2c-ac5b06d1e741 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.222819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a1d5d-2264-4c4f-8a2c-ac5b06d1e741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-834a1d5d-2264-4c4f-8a2c-ac5b06d1e741 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:40.230804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:40.233190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:40.234815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 422/835] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:20:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:40.241660 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5400431-b197-4f9a-b97d-a1154c79cd93 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:40.362893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5400431-b197-4f9a-b97d-a1154c79cd93 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:40.401153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba69e5-dd89-4aa4-8a61-736a6ad7cb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:40.401949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba69e5-dd89-4aa4-8a61-736a6ad7cb1b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.402206 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba69e5-dd89-4aa4-8a61-736a6ad7cb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.402469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba69e5-dd89-4aa4-8a61-736a6ad7cb1b None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.403068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba69e5-dd89-4aa4-8a61-736a6ad7cb1b None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:40.403384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba69e5-dd89-4aa4-8a61-736a6ad7cb1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:40.404792 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba69e5-dd89-4aa4-8a61-736a6ad7cb1b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:40.423649 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 414/836] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:20:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:40.433089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a1d5d-2264-4c4f-8a2c-ac5b06d1e741 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:40.496891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:40.557276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a1d5d-2264-4c4f-8a2c-ac5b06d1e741 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:40.715981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce10b55-df06-4b8c-bd10-a7cb903cedca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:40.716749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10b55-df06-4b8c-bd10-a7cb903cedca None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.717083 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10b55-df06-4b8c-bd10-a7cb903cedca None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.717357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10b55-df06-4b8c-bd10-a7cb903cedca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.728754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce10b55-df06-4b8c-bd10-a7cb903cedca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:40.730968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce10b55-df06-4b8c-bd10-a7cb903cedca None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-2ce10b55-df06-4b8c-bd10-a7cb903cedca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:40.753283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:40.755411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:40.757535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:40.757535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:40.757535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:40.757535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:40.757535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 423/837] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:40.765473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba69e5-dd89-4aa4-8a61-736a6ad7cb1b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:40.790880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:40.876522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625d3e9-7f5e-431e-8e2d-fdd566f92f6e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.876864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625d3e9-7f5e-431e-8e2d-fdd566f92f6e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.877056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625d3e9-7f5e-431e-8e2d-fdd566f92f6e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.878756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 424/838] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:40.889006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b33229-a96e-438d-8aa0-53dc6ef0868a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.889553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b33229-a96e-438d-8aa0-53dc6ef0868a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.890037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b33229-a96e-438d-8aa0-53dc6ef0868a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.923095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-66b33229-a96e-438d-8aa0-53dc6ef0868a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:40.923847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba69e5-dd89-4aa4-8a61-736a6ad7cb1b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:40.925334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-66b33229-a96e-438d-8aa0-53dc6ef0868a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:40.965914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4678442-5f87-4fed-8be8-ba40c49bba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:40.967315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4678442-5f87-4fed-8be8-ba40c49bba34 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:40.967540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4678442-5f87-4fed-8be8-ba40c49bba34 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:40.967826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4678442-5f87-4fed-8be8-ba40c49bba34 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:40.968775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4678442-5f87-4fed-8be8-ba40c49bba34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:40.969228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4678442-5f87-4fed-8be8-ba40c49bba34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:40.970623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4678442-5f87-4fed-8be8-ba40c49bba34 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:40.987304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 415/839] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:40.996961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4678442-5f87-4fed-8be8-ba40c49bba34 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.037472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 425/840] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:41.043395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aba084-328d-4acb-83b0-b13ff711c9aa None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.043704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aba084-328d-4acb-83b0-b13ff711c9aa None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.043938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aba084-328d-4acb-83b0-b13ff711c9aa None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.049540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 426/841] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:41.050907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f38859-9edc-4322-b06d-9c10c2166615 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.051189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f38859-9edc-4322-b06d-9c10c2166615 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.051462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f38859-9edc-4322-b06d-9c10c2166615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.069906 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f3f38859-9edc-4322-b06d-9c10c2166615 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:41.071115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f3f38859-9edc-4322-b06d-9c10c2166615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:41.095610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:41.118321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:41.152717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 427/842] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:41.152717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f38859-9edc-4322-b06d-9c10c2166615 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.171277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4678442-5f87-4fed-8be8-ba40c49bba34 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.217283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feea532-945c-4c0b-ae89-376c0b1e867f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.218191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feea532-945c-4c0b-ae89-376c0b1e867f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.218440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feea532-945c-4c0b-ae89-376c0b1e867f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.218721 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feea532-945c-4c0b-ae89-376c0b1e867f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.227957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feea532-945c-4c0b-ae89-376c0b1e867f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.230049 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feea532-945c-4c0b-ae89-376c0b1e867f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-7feea532-945c-4c0b-ae89-376c0b1e867f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:41.246070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:41.248254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:41.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:41.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:41.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:41.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:41.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:41.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:41.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:41.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:41.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:41.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:41.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 416/843] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 22:20:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:41.249623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feea532-945c-4c0b-ae89-376c0b1e867f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.311829 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f38859-9edc-4322-b06d-9c10c2166615 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.352285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06375c98-6425-40da-88a2-c71d9131ed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.353144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06375c98-6425-40da-88a2-c71d9131ed32 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.353430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06375c98-6425-40da-88a2-c71d9131ed32 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.354529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06375c98-6425-40da-88a2-c71d9131ed32 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.354964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06375c98-6425-40da-88a2-c71d9131ed32 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.356372 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06375c98-6425-40da-88a2-c71d9131ed32 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-06375c98-6425-40da-88a2-c71d9131ed32 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:41.370065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:41.373289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:41.374597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 428/844] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Jul 21 22:20:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:41.376947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7eb829-d9f6-4c24-b079-a4be33851824 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.376947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7eb829-d9f6-4c24-b079-a4be33851824 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.376947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7eb829-d9f6-4c24-b079-a4be33851824 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.376947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 429/845] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:41.376947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7eb829-d9f6-4c24-b079-a4be33851824 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.385867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feea532-945c-4c0b-ae89-376c0b1e867f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.399393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:41.436275 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:41.443099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf2a47-ea4b-43bc-a3f6-2faf46358604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.443908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf2a47-ea4b-43bc-a3f6-2faf46358604 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.444199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf2a47-ea4b-43bc-a3f6-2faf46358604 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.444505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf2a47-ea4b-43bc-a3f6-2faf46358604 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.456828 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf2a47-ea4b-43bc-a3f6-2faf46358604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-12bf2a47-ea4b-43bc-a3f6-2faf46358604 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:41.466111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:41.467197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 417/846] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:41.468040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf2a47-ea4b-43bc-a3f6-2faf46358604 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.501402 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7eb829-d9f6-4c24-b079-a4be33851824 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.544082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8dd77-f55c-44c0-bae5-9c9af4c02cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.545295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8dd77-f55c-44c0-bae5-9c9af4c02cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.545788 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8dd77-f55c-44c0-bae5-9c9af4c02cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.546304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8dd77-f55c-44c0-bae5-9c9af4c02cd6 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.547931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8dd77-f55c-44c0-bae5-9c9af4c02cd6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:41.548524 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8dd77-f55c-44c0-bae5-9c9af4c02cd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.551377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8dd77-f55c-44c0-bae5-9c9af4c02cd6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:41.579412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 430/847] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jul 21 22:20:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:41.581454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fb0e5-3f10-493d-a2f7-23781d84e82c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.581856 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fb0e5-3f10-493d-a2f7-23781d84e82c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.582310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fb0e5-3f10-493d-a2f7-23781d84e82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.595028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf2a47-ea4b-43bc-a3f6-2faf46358604 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.603831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a47fb0e5-3f10-493d-a2f7-23781d84e82c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:41.605295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a47fb0e5-3f10-493d-a2f7-23781d84e82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:41.636500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3bc21-7895-40dd-875c-a8a1e9e09fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.637183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3bc21-7895-40dd-875c-a8a1e9e09fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.637468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3bc21-7895-40dd-875c-a8a1e9e09fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.637742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3bc21-7895-40dd-875c-a8a1e9e09fe2 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.638467 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3bc21-7895-40dd-875c-a8a1e9e09fe2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:41.638775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3bc21-7895-40dd-875c-a8a1e9e09fe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.640154 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3bc21-7895-40dd-875c-a8a1e9e09fe2 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:41.652646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 418/848] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:41.652646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3bc21-7895-40dd-875c-a8a1e9e09fe2 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.693256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 431/849] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:41.694621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fb0e5-3f10-493d-a2f7-23781d84e82c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.710405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:41.719066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:41.774404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3bc21-7895-40dd-875c-a8a1e9e09fe2 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.820647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f70875-91f0-4955-9f1c-302483ac5752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.821650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f70875-91f0-4955-9f1c-302483ac5752 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.822036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f70875-91f0-4955-9f1c-302483ac5752 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.823016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f70875-91f0-4955-9f1c-302483ac5752 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.824327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f70875-91f0-4955-9f1c-302483ac5752 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:41.824850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f70875-91f0-4955-9f1c-302483ac5752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.826544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f70875-91f0-4955-9f1c-302483ac5752 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:41.832509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fb0e5-3f10-493d-a2f7-23781d84e82c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.840142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 419/850] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:41.841230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f70875-91f0-4955-9f1c-302483ac5752 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:41.884623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaba0c4-6fdb-4a1e-b8b5-6aa4d738ac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:41.886885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaba0c4-6fdb-4a1e-b8b5-6aa4d738ac90 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:41.887335 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaba0c4-6fdb-4a1e-b8b5-6aa4d738ac90 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:41.887794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaba0c4-6fdb-4a1e-b8b5-6aa4d738ac90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:41.985659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f70875-91f0-4955-9f1c-302483ac5752 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:41.988583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaba0c4-6fdb-4a1e-b8b5-6aa4d738ac90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:41.990464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaba0c4-6fdb-4a1e-b8b5-6aa4d738ac90 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:42.005345 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:42.008259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:42.034612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d73bd4-1638-445f-a494-aa5e91891d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:42.035655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d73bd4-1638-445f-a494-aa5e91891d21 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:42.036108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d73bd4-1638-445f-a494-aa5e91891d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:42.036617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d73bd4-1638-445f-a494-aa5e91891d21 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:42.037789 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d73bd4-1638-445f-a494-aa5e91891d21 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:42.038220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d73bd4-1638-445f-a494-aa5e91891d21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:42.040019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d73bd4-1638-445f-a494-aa5e91891d21 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:42.046152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 420/851] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:42.046983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d73bd4-1638-445f-a494-aa5e91891d21 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:42.132785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 432/852] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:41 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:20:42.134723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaba0c4-6fdb-4a1e-b8b5-6aa4d738ac90 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:42.216193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d73bd4-1638-445f-a494-aa5e91891d21 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:42.265415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d7d7f-5fc1-4f81-b444-a021e751e670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:42.266235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d7d7f-5fc1-4f81-b444-a021e751e670 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:42.266523 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d7d7f-5fc1-4f81-b444-a021e751e670 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:42.266800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d7d7f-5fc1-4f81-b444-a021e751e670 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:42.267463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d7d7f-5fc1-4f81-b444-a021e751e670 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:42.269627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d7d7f-5fc1-4f81-b444-a021e751e670 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-176d7d7f-5fc1-4f81-b444-a021e751e670 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:42.283187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:42.285204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:42.287968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 421/853] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:42.292306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d7d7f-5fc1-4f81-b444-a021e751e670 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:42.328046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaba0c4-6fdb-4a1e-b8b5-6aa4d738ac90 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:42.343563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:42.347976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:42.388329 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb33807-6bc7-4101-b6e6-5ff41efc4f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:42.389742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb33807-6bc7-4101-b6e6-5ff41efc4f06 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:42.390231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb33807-6bc7-4101-b6e6-5ff41efc4f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:42.390723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb33807-6bc7-4101-b6e6-5ff41efc4f06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:42.407698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb33807-6bc7-4101-b6e6-5ff41efc4f06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:42.409248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb33807-6bc7-4101-b6e6-5ff41efc4f06 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:42.417187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d7d7f-5fc1-4f81-b444-a021e751e670 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-7fb33807-6bc7-4101-b6e6-5ff41efc4f06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:42.430816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:42.432430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:42.433946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:42.433946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:42.433946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:42.433946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:42.437676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 433/854] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:42.438522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb33807-6bc7-4101-b6e6-5ff41efc4f06 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:42.467893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bfff17-c6ca-437f-bc8f-476f5c39614f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:42.468841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfff17-c6ca-437f-bc8f-476f5c39614f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:42.469222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfff17-c6ca-437f-bc8f-476f5c39614f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:42.469375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfff17-c6ca-437f-bc8f-476f5c39614f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:42.483156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bfff17-c6ca-437f-bc8f-476f5c39614f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:42.484719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bfff17-c6ca-437f-bc8f-476f5c39614f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-18bfff17-c6ca-437f-bc8f-476f5c39614f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:42.511466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:42.513213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:42.514903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:42.514903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:42.514903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:42.514903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:42.516259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 422/855] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:42.517854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bfff17-c6ca-437f-bc8f-476f5c39614f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:42.557601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb33807-6bc7-4101-b6e6-5ff41efc4f06 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:42.599075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05bc6d-0fa4-40c7-af2f-4f5441d3c0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:42.600179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05bc6d-0fa4-40c7-af2f-4f5441d3c0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:42.600700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05bc6d-0fa4-40c7-af2f-4f5441d3c0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:42.601192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05bc6d-0fa4-40c7-af2f-4f5441d3c0c9 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:42.602506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05bc6d-0fa4-40c7-af2f-4f5441d3c0c9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:42.603056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05bc6d-0fa4-40c7-af2f-4f5441d3c0c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:42.606351 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05bc6d-0fa4-40c7-af2f-4f5441d3c0c9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:42.614918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 434/856] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:42.623789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05bc6d-0fa4-40c7-af2f-4f5441d3c0c9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:42.652662 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:42.672370 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:42.679816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bfff17-c6ca-437f-bc8f-476f5c39614f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:42.731384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d12f11-c0aa-4800-928d-c31c985dd660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:42.732455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d12f11-c0aa-4800-928d-c31c985dd660 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:42.732980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d12f11-c0aa-4800-928d-c31c985dd660 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:42.733456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d12f11-c0aa-4800-928d-c31c985dd660 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:42.734383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d12f11-c0aa-4800-928d-c31c985dd660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:42.734876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d12f11-c0aa-4800-928d-c31c985dd660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:42.736448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d12f11-c0aa-4800-928d-c31c985dd660 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:42.750072 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 423/857] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:42.751347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d12f11-c0aa-4800-928d-c31c985dd660 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:42.770341 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05bc6d-0fa4-40c7-af2f-4f5441d3c0c9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:42.828268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b7c2c-b73f-4576-9933-7ae9aed6e043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:42.829135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b7c2c-b73f-4576-9933-7ae9aed6e043 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:42.829589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b7c2c-b73f-4576-9933-7ae9aed6e043 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:42.830043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b7c2c-b73f-4576-9933-7ae9aed6e043 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:42.831289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b7c2c-b73f-4576-9933-7ae9aed6e043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:42.831799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b7c2c-b73f-4576-9933-7ae9aed6e043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:42.834127 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b7c2c-b73f-4576-9933-7ae9aed6e043 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:42.856973 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 435/858] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:42.856973 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b7c2c-b73f-4576-9933-7ae9aed6e043 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:42.914420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d12f11-c0aa-4800-928d-c31c985dd660 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:42.960312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf21647-b8c5-4487-bbac-9807211e8abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:42.961519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf21647-b8c5-4487-bbac-9807211e8abb None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:42.961948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf21647-b8c5-4487-bbac-9807211e8abb None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:42.962369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf21647-b8c5-4487-bbac-9807211e8abb None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:42.964089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf21647-b8c5-4487-bbac-9807211e8abb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:42.964660 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf21647-b8c5-4487-bbac-9807211e8abb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:42.965842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:42.967920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf21647-b8c5-4487-bbac-9807211e8abb None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:42.975034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 424/859] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:42.976117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf21647-b8c5-4487-bbac-9807211e8abb None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:42.999713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b7c2c-b73f-4576-9933-7ae9aed6e043 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:43.024536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:43.067913 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f3a83-38f9-4916-9b95-90e0a58dc920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:43.068938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f3a83-38f9-4916-9b95-90e0a58dc920 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:43.069419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f3a83-38f9-4916-9b95-90e0a58dc920 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:43.069895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f3a83-38f9-4916-9b95-90e0a58dc920 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:43.084450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f3a83-38f9-4916-9b95-90e0a58dc920 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:43.086962 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f3a83-38f9-4916-9b95-90e0a58dc920 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-c04f3a83-38f9-4916-9b95-90e0a58dc920 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:43.116925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:43.118803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:43.121336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:43.121336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:43.121336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:43.121336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:43.121336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:43.121336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:43.121336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:43.121336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:43.121336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:20:43.121336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:43.121336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 436/860] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:20:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:43.125445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f3a83-38f9-4916-9b95-90e0a58dc920 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:43.191074 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf21647-b8c5-4487-bbac-9807211e8abb None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:43.235082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e65c4-6da0-45ec-aa8d-fee55a98b61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:43.236028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e65c4-6da0-45ec-aa8d-fee55a98b61c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:43.236314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e65c4-6da0-45ec-aa8d-fee55a98b61c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:43.236631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e65c4-6da0-45ec-aa8d-fee55a98b61c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:43.245270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e65c4-6da0-45ec-aa8d-fee55a98b61c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:43.246890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e65c4-6da0-45ec-aa8d-fee55a98b61c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-e97e65c4-6da0-45ec-aa8d-fee55a98b61c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:43.263296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:43.264451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:43.265963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:43.265963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:43.265963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:43.265963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:43.265963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:43.265963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:43.265963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:43.265963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:43.265963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 425/861] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:43.270856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e65c4-6da0-45ec-aa8d-fee55a98b61c None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:43.292300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f3a83-38f9-4916-9b95-90e0a58dc920 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:43.342980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c58d5-e642-4d67-8997-73f41321fe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:43.343914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c58d5-e642-4d67-8997-73f41321fe34 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:43.344662 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c58d5-e642-4d67-8997-73f41321fe34 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:43.345125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c58d5-e642-4d67-8997-73f41321fe34 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:43.346981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c58d5-e642-4d67-8997-73f41321fe34 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:43.347478 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c58d5-e642-4d67-8997-73f41321fe34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:43.349840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c58d5-e642-4d67-8997-73f41321fe34 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:43.377524 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 437/862] 10.222.0.42 () {56 vars in 1135 bytes} [Thu Jul 21 22:20:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:43.388919 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c58d5-e642-4d67-8997-73f41321fe34 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:43.389896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:43.396845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:43.457594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e65c4-6da0-45ec-aa8d-fee55a98b61c None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:43.505659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c58d5-e642-4d67-8997-73f41321fe34 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:43.527125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811ec16-8b53-43c5-bc79-cf737ae22186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:43.528822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811ec16-8b53-43c5-bc79-cf737ae22186 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:43.529642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811ec16-8b53-43c5-bc79-cf737ae22186 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:43.530442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811ec16-8b53-43c5-bc79-cf737ae22186 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:43.532357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811ec16-8b53-43c5-bc79-cf737ae22186 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:43.533260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811ec16-8b53-43c5-bc79-cf737ae22186 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:43.536078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811ec16-8b53-43c5-bc79-cf737ae22186 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:43.547684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088385e-2a02-4156-940c-5a8bf0489acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:43.548353 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088385e-2a02-4156-940c-5a8bf0489acb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:43.548727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088385e-2a02-4156-940c-5a8bf0489acb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:43.548999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088385e-2a02-4156-940c-5a8bf0489acb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:43.564194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 426/863] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:43.574247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811ec16-8b53-43c5-bc79-cf737ae22186 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:43.642521 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088385e-2a02-4156-940c-5a8bf0489acb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:43.644924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088385e-2a02-4156-940c-5a8bf0489acb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:43.646272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:43.764515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 438/864] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:20:43.770138 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811ec16-8b53-43c5-bc79-cf737ae22186 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:43.770932 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088385e-2a02-4156-940c-5a8bf0489acb None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:43.819852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbde8f-43ba-4f89-b635-c5ee6e341d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:43.821429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbde8f-43ba-4f89-b635-c5ee6e341d21 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:43.821817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbde8f-43ba-4f89-b635-c5ee6e341d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:43.822264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbde8f-43ba-4f89-b635-c5ee6e341d21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:43.824657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbde8f-43ba-4f89-b635-c5ee6e341d21 None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=d53f6748398d45358f7b2d75ec273dd6) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:43.825123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbde8f-43ba-4f89-b635-c5ee6e341d21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:43.826699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbde8f-43ba-4f89-b635-c5ee6e341d21 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:43.840716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 427/865] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:43 2022] GET /identity/v3/role_assignments?user.id=d53f6748398d45358f7b2d75ec273dd6&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:43.947836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088385e-2a02-4156-940c-5a8bf0489acb None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:43.950428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:44.003657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b6e24-c69f-4df1-9569-015660ccea99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:44.006324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b6e24-c69f-4df1-9569-015660ccea99 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.007082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b6e24-c69f-4df1-9569-015660ccea99 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.007826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b6e24-c69f-4df1-9569-015660ccea99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.029625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b6e24-c69f-4df1-9569-015660ccea99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:44.031420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7b6e24-c69f-4df1-9569-015660ccea99 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-9d7b6e24-c69f-4df1-9569-015660ccea99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:44.051606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:44.052661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:44.053595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:44.053595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:44.053595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:44.053595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:44.053595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 439/866] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:44.059697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbde8f-43ba-4f89-b635-c5ee6e341d21 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:44.064583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:44.255393 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbde8f-43ba-4f89-b635-c5ee6e341d21 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:44.293373 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c82d5-3abf-43dd-80c0-d20f8b8d5cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:44.294457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c82d5-3abf-43dd-80c0-d20f8b8d5cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.294952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c82d5-3abf-43dd-80c0-d20f8b8d5cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.295437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c82d5-3abf-43dd-80c0-d20f8b8d5cd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.296395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c82d5-3abf-43dd-80c0-d20f8b8d5cd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:44.297059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c82d5-3abf-43dd-80c0-d20f8b8d5cd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:44.298755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c82d5-3abf-43dd-80c0-d20f8b8d5cd8 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:44.313599 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 428/867] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:44.319043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b6e24-c69f-4df1-9569-015660ccea99 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:44.448054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b6e24-c69f-4df1-9569-015660ccea99 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:44.487343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593eaace-d741-4866-b4ad-06039a16d854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:44.488545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593eaace-d741-4866-b4ad-06039a16d854 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.489044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593eaace-d741-4866-b4ad-06039a16d854 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.489526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593eaace-d741-4866-b4ad-06039a16d854 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.491540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5392d30-dd2a-4f19-a04c-6d9dba5e1400 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.492086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5392d30-dd2a-4f19-a04c-6d9dba5e1400 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.492793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5392d30-dd2a-4f19-a04c-6d9dba5e1400 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.495532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 429/868] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:44.502125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593eaace-d741-4866-b4ad-06039a16d854 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:44.502602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926362f-a227-41b9-872f-7742829ef3be None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.503130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926362f-a227-41b9-872f-7742829ef3be None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.503612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926362f-a227-41b9-872f-7742829ef3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.503790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593eaace-d741-4866-b4ad-06039a16d854 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:44.523197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7926362f-a227-41b9-872f-7742829ef3be None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:44.524803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7926362f-a227-41b9-872f-7742829ef3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-593eaace-d741-4866-b4ad-06039a16d854 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:44.525084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:44.526519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:44.527443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:44.527443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:44.527443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:44.527443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:44.527443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:44.527443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:44.527443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:44.527443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:44.527888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 440/869] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:44.533852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593eaace-d741-4866-b4ad-06039a16d854 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:44.554855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:44.646495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:44.659832 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593eaace-d741-4866-b4ad-06039a16d854 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:44.671326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 430/870] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:44.694422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926362f-a227-41b9-872f-7742829ef3be None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:44.710820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399e55c-d618-4233-a404-22f13c378e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:44.712552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399e55c-d618-4233-a404-22f13c378e39 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.713847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399e55c-d618-4233-a404-22f13c378e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.714461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399e55c-d618-4233-a404-22f13c378e39 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.715981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399e55c-d618-4233-a404-22f13c378e39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:44.717087 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399e55c-d618-4233-a404-22f13c378e39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:44.719156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399e55c-d618-4233-a404-22f13c378e39 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:44.743564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 441/871] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:44.751208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399e55c-d618-4233-a404-22f13c378e39 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:44.829827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926362f-a227-41b9-872f-7742829ef3be None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:44.848365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:44.893107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6981a18-d2dd-4f47-add8-f72169b65339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:44.894389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6981a18-d2dd-4f47-add8-f72169b65339 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.894859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6981a18-d2dd-4f47-add8-f72169b65339 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.895417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6981a18-d2dd-4f47-add8-f72169b65339 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.896338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6981a18-d2dd-4f47-add8-f72169b65339 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:44.897928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6981a18-d2dd-4f47-add8-f72169b65339 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:44.898780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399e55c-d618-4233-a404-22f13c378e39 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:44.922773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 431/872] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:20:44 2022] POST /identity/v3/services => generated 269 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:20:44.946795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52802df8-43d0-475d-b4af-1677d9e82bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:44.948052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52802df8-43d0-475d-b4af-1677d9e82bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:44.948799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52802df8-43d0-475d-b4af-1677d9e82bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:44.948799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52802df8-43d0-475d-b4af-1677d9e82bd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:44.949472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52802df8-43d0-475d-b4af-1677d9e82bd5 None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=8fec30ef0b8c41dca643cfefebfe904b) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:44.949784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52802df8-43d0-475d-b4af-1677d9e82bd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:44.951320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52802df8-43d0-475d-b4af-1677d9e82bd5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:44.964724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 442/873] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:44 2022] GET /identity/v3/role_assignments?user.id=8fec30ef0b8c41dca643cfefebfe904b&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=f957e09d047b47cfa0891d25b4181263 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:45.272122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e8044-8cdf-4b0f-8e4a-279c0053283b None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:45.273313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e8044-8cdf-4b0f-8e4a-279c0053283b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:45.273822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e8044-8cdf-4b0f-8e4a-279c0053283b None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:45.276842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 432/874] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:45.282838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae3632-284c-4b28-b08a-ea5857e94350 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:45.283829 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae3632-284c-4b28-b08a-ea5857e94350 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:45.284310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae3632-284c-4b28-b08a-ea5857e94350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:45.303398 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-73ae3632-284c-4b28-b08a-ea5857e94350 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:45.305046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-73ae3632-284c-4b28-b08a-ea5857e94350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:45.392933 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 443/875] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:45 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:45.419380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e8044-8cdf-4b0f-8e4a-279c0053283b None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:45.502993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:45.553977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e8044-8cdf-4b0f-8e4a-279c0053283b None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:45.596642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bead6-3acc-4880-a2e9-55149291d932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:45.597646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bead6-3acc-4880-a2e9-55149291d932 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:45.598115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bead6-3acc-4880-a2e9-55149291d932 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:45.598573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bead6-3acc-4880-a2e9-55149291d932 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:45.599531 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bead6-3acc-4880-a2e9-55149291d932 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:45.601450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bead6-3acc-4880-a2e9-55149291d932 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:45.622608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 433/876] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:20:45 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:46.124496 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893096a8-f35c-419a-a98c-c0ba59cec033 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:46.124882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893096a8-f35c-419a-a98c-c0ba59cec033 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:46.125163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893096a8-f35c-419a-a98c-c0ba59cec033 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:46.126716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 444/877] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:46.132242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb246e-0706-477c-8866-94f4cd065acd None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:46.132561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb246e-0706-477c-8866-94f4cd065acd None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:46.132844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb246e-0706-477c-8866-94f4cd065acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:46.161646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-1feb246e-0706-477c-8866-94f4cd065acd None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:46.163765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1feb246e-0706-477c-8866-94f4cd065acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:46.285043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 434/878] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:46.323434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893096a8-f35c-419a-a98c-c0ba59cec033 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:46.405760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:46.463899 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893096a8-f35c-419a-a98c-c0ba59cec033 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:46.508138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db969e92-7182-41dd-8bb9-139380e495db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:46.509033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db969e92-7182-41dd-8bb9-139380e495db None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:46.509317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db969e92-7182-41dd-8bb9-139380e495db None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:46.509632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db969e92-7182-41dd-8bb9-139380e495db None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:46.523381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db969e92-7182-41dd-8bb9-139380e495db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-db969e92-7182-41dd-8bb9-139380e495db None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:46.532120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:46.533245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:46.534119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 445/879] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jul 21 22:20:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:46.542278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb246e-0706-477c-8866-94f4cd065acd None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:46.604922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:46.665325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb246e-0706-477c-8866-94f4cd065acd None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:46.701820 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef0744-99b1-4efd-9453-e8205b06d05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:46.702493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef0744-99b1-4efd-9453-e8205b06d05c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:46.702777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef0744-99b1-4efd-9453-e8205b06d05c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:46.703050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef0744-99b1-4efd-9453-e8205b06d05c None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:46.703820 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef0744-99b1-4efd-9453-e8205b06d05c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:46.704120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef0744-99b1-4efd-9453-e8205b06d05c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:46.705652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef0744-99b1-4efd-9453-e8205b06d05c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:46.715775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 435/880] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 22:20:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:46.721592 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db969e92-7182-41dd-8bb9-139380e495db None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:46.853259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db969e92-7182-41dd-8bb9-139380e495db None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:46.882986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:46.895626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db90946-1e65-493f-b6df-23c944a5f7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:46.896432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db90946-1e65-493f-b6df-23c944a5f7ac None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:46.896752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db90946-1e65-493f-b6df-23c944a5f7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:46.897028 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db90946-1e65-493f-b6df-23c944a5f7ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:46.909618 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db90946-1e65-493f-b6df-23c944a5f7ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:46.911109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db90946-1e65-493f-b6df-23c944a5f7ac None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-5db90946-1e65-493f-b6df-23c944a5f7ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:46.930117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:46.932237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:46.934393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:46.934393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:46.934393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:46.934393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:46.934393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 446/881] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:46.937284 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef0744-99b1-4efd-9453-e8205b06d05c None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:47.057037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef0744-99b1-4efd-9453-e8205b06d05c None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:47.058815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:47.103053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397b9f5-9a7f-46ef-9768-45bb622dabd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:47.104508 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397b9f5-9a7f-46ef-9768-45bb622dabd6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:47.105001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397b9f5-9a7f-46ef-9768-45bb622dabd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:47.105279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397b9f5-9a7f-46ef-9768-45bb622dabd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:47.106459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397b9f5-9a7f-46ef-9768-45bb622dabd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:47.106833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397b9f5-9a7f-46ef-9768-45bb622dabd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:47.108219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397b9f5-9a7f-46ef-9768-45bb622dabd6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:47.127474 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 436/882] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:47.133557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db90946-1e65-493f-b6df-23c944a5f7ac None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:47.219417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d88e0-6c0b-4923-a0ad-3421279c52c5 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:47.220637 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d88e0-6c0b-4923-a0ad-3421279c52c5 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:47.220637 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d88e0-6c0b-4923-a0ad-3421279c52c5 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:47.224350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 437/883] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:47.235376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3960b8-73a4-43d6-8801-49dc7114032c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:47.236031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3960b8-73a4-43d6-8801-49dc7114032c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:47.236457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3960b8-73a4-43d6-8801-49dc7114032c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:47.267331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-0c3960b8-73a4-43d6-8801-49dc7114032c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:47.269348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0c3960b8-73a4-43d6-8801-49dc7114032c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:47.276402 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db90946-1e65-493f-b6df-23c944a5f7ac None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:47.334239 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e7f4b-1a0d-4eb8-8ec5-8bff796447fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:47.335593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e7f4b-1a0d-4eb8-8ec5-8bff796447fe None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:47.335998 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e7f4b-1a0d-4eb8-8ec5-8bff796447fe None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:47.336456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e7f4b-1a0d-4eb8-8ec5-8bff796447fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:47.349512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e7f4b-1a0d-4eb8-8ec5-8bff796447fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:47.351871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e7f4b-1a0d-4eb8-8ec5-8bff796447fe None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e52e7f4b-1a0d-4eb8-8ec5-8bff796447fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:47.373506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:47.374540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:47.376173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:47.376173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:47.376173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:47.376173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:47.376173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:47.376173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:47.376173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:47.376173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:47.376173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:47.376173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:47.376173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 447/884] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 22:20:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:47.377045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e28c5-e5dc-48c8-8a95-942dc00f629f None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:47.377045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e28c5-e5dc-48c8-8a95-942dc00f629f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:47.377045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e28c5-e5dc-48c8-8a95-942dc00f629f None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:47.382469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 448/885] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:47.382620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 438/886] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:47.385506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e28c5-e5dc-48c8-8a95-942dc00f629f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:47.386745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdad0d5-f2bf-45f4-830b-27e1e3ee34b5 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:47.387274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdad0d5-f2bf-45f4-830b-27e1e3ee34b5 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:47.387761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdad0d5-f2bf-45f4-830b-27e1e3ee34b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:47.407038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-3cdad0d5-f2bf-45f4-830b-27e1e3ee34b5 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:47.408726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3cdad0d5-f2bf-45f4-830b-27e1e3ee34b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:47.443507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:47.516590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 439/887] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:47.516590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdad0d5-f2bf-45f4-830b-27e1e3ee34b5 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:47.543850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e28c5-e5dc-48c8-8a95-942dc00f629f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:47.547053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:47.586587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6e6da-4622-49cc-84c6-d75413152361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:47.587293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6e6da-4622-49cc-84c6-d75413152361 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:47.587618 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6e6da-4622-49cc-84c6-d75413152361 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:47.587903 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6e6da-4622-49cc-84c6-d75413152361 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:47.588794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6e6da-4622-49cc-84c6-d75413152361 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:47.589111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6e6da-4622-49cc-84c6-d75413152361 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:47.590623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6e6da-4622-49cc-84c6-d75413152361 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:47.613172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 449/888] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jul 21 22:20:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:47.614471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6e6da-4622-49cc-84c6-d75413152361 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:47.638214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdad0d5-f2bf-45f4-830b-27e1e3ee34b5 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:47.677075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5b491-9073-4f26-835a-ca50cfd820ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:47.678022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5b491-9073-4f26-835a-ca50cfd820ed None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:47.678312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5b491-9073-4f26-835a-ca50cfd820ed None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:47.678588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5b491-9073-4f26-835a-ca50cfd820ed None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:47.679259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5b491-9073-4f26-835a-ca50cfd820ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:47.680647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5b491-9073-4f26-835a-ca50cfd820ed None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-f2c5b491-9073-4f26-835a-ca50cfd820ed None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:47.693714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:47.694738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:47.695614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:47.700334 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 440/889] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Jul 21 22:20:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:47.701265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5b491-9073-4f26-835a-ca50cfd820ed None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:47.729826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:47.738445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6e6da-4622-49cc-84c6-d75413152361 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:47.779310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23728ad8-e73e-4176-aac5-e11ae1f76885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:47.780492 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23728ad8-e73e-4176-aac5-e11ae1f76885 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:47.780788 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23728ad8-e73e-4176-aac5-e11ae1f76885 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:47.781061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23728ad8-e73e-4176-aac5-e11ae1f76885 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:47.782080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23728ad8-e73e-4176-aac5-e11ae1f76885 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:47.783855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23728ad8-e73e-4176-aac5-e11ae1f76885 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:47.790595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-23728ad8-e73e-4176-aac5-e11ae1f76885 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:47.797485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:47.798540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:47.799436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 450/890] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:47.801238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23728ad8-e73e-4176-aac5-e11ae1f76885 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:47.812772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5b491-9073-4f26-835a-ca50cfd820ed None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:47.848824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134cbed-a4ae-4717-b03d-aab9736e760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:47.849487 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134cbed-a4ae-4717-b03d-aab9736e760f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:47.849779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134cbed-a4ae-4717-b03d-aab9736e760f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:47.850057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134cbed-a4ae-4717-b03d-aab9736e760f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:47.905474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23728ad8-e73e-4176-aac5-e11ae1f76885 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:47.920817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134cbed-a4ae-4717-b03d-aab9736e760f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:47.922265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134cbed-a4ae-4717-b03d-aab9736e760f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:47.938380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf05a88-378f-42eb-95d4-6ce124e0389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:47.939051 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf05a88-378f-42eb-95d4-6ce124e0389e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:47.939331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf05a88-378f-42eb-95d4-6ce124e0389e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:47.939604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf05a88-378f-42eb-95d4-6ce124e0389e None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:47.940307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf05a88-378f-42eb-95d4-6ce124e0389e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:47.940649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf05a88-378f-42eb-95d4-6ce124e0389e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:47.941997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf05a88-378f-42eb-95d4-6ce124e0389e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:47.946519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 451/891] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:47.947822 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf05a88-378f-42eb-95d4-6ce124e0389e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:47.972153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:48.008894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:48.026587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 441/892] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:47 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:20:48.027927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437bf65-884c-4105-ab1c-e05e49a2c0de None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.028226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437bf65-884c-4105-ab1c-e05e49a2c0de None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.028533 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437bf65-884c-4105-ab1c-e05e49a2c0de None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.029856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 442/893] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:48.030502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437bf65-884c-4105-ab1c-e05e49a2c0de None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:48.052618 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf05a88-378f-42eb-95d4-6ce124e0389e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:48.088362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c01539-fb7d-4241-9220-82fdbd34bb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:48.089118 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c01539-fb7d-4241-9220-82fdbd34bb75 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.089402 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c01539-fb7d-4241-9220-82fdbd34bb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.089671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c01539-fb7d-4241-9220-82fdbd34bb75 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.090370 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c01539-fb7d-4241-9220-82fdbd34bb75 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:48.090669 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c01539-fb7d-4241-9220-82fdbd34bb75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:48.092035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c01539-fb7d-4241-9220-82fdbd34bb75 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:48.096638 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 452/894] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:48.097423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c01539-fb7d-4241-9220-82fdbd34bb75 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:48.139787 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437bf65-884c-4105-ab1c-e05e49a2c0de None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:48.175065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df2976-d9c8-4d0c-a283-5848d32c59cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:48.175733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df2976-d9c8-4d0c-a283-5848d32c59cd None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.176018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df2976-d9c8-4d0c-a283-5848d32c59cd None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.176305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df2976-d9c8-4d0c-a283-5848d32c59cd None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.177054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df2976-d9c8-4d0c-a283-5848d32c59cd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:48.177367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df2976-d9c8-4d0c-a283-5848d32c59cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:48.178742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df2976-d9c8-4d0c-a283-5848d32c59cd None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:48.182890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:48.184715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 443/895] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:48.185987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd6d76-12aa-4e12-8803-aeca0e0be89e None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.186282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd6d76-12aa-4e12-8803-aeca0e0be89e None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.186563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd6d76-12aa-4e12-8803-aeca0e0be89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.199369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c01539-fb7d-4241-9220-82fdbd34bb75 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:48.202927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-66cd6d76-12aa-4e12-8803-aeca0e0be89e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:48.204135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-66cd6d76-12aa-4e12-8803-aeca0e0be89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:48.238871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00604394-1e0f-4039-9cf9-453aa64737b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:48.239674 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00604394-1e0f-4039-9cf9-453aa64737b5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.239962 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00604394-1e0f-4039-9cf9-453aa64737b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.240237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00604394-1e0f-4039-9cf9-453aa64737b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.251087 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00604394-1e0f-4039-9cf9-453aa64737b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:48.251995 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:48.252580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00604394-1e0f-4039-9cf9-453aa64737b5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-00604394-1e0f-4039-9cf9-453aa64737b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:48.271321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:48.272606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:48.273874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:48.273874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:48.273874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:48.273874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:48.273874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 453/896] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:48.273874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00604394-1e0f-4039-9cf9-453aa64737b5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:48.292855 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 444/897] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:48.292855 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd6d76-12aa-4e12-8803-aeca0e0be89e None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:48.375844 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00604394-1e0f-4039-9cf9-453aa64737b5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:48.377146 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd6d76-12aa-4e12-8803-aeca0e0be89e None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:48.411259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f86cad-5ec1-4ec4-95f5-581e3892383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:48.411810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40cb70-13d3-4fb3-bf05-efdcc9198862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:48.412265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f86cad-5ec1-4ec4-95f5-581e3892383c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.412265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f86cad-5ec1-4ec4-95f5-581e3892383c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.412448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f86cad-5ec1-4ec4-95f5-581e3892383c None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.412571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40cb70-13d3-4fb3-bf05-efdcc9198862 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.412800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40cb70-13d3-4fb3-bf05-efdcc9198862 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.413084 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40cb70-13d3-4fb3-bf05-efdcc9198862 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.413191 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f86cad-5ec1-4ec4-95f5-581e3892383c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:48.413461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f86cad-5ec1-4ec4-95f5-581e3892383c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:48.413887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40cb70-13d3-4fb3-bf05-efdcc9198862 None admin] RBAC: Adding query filter params (interface=public, service_id=d6abd1e0a24443f19918c8a9567fbc9f, region_id=RegionOne) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:48.414207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40cb70-13d3-4fb3-bf05-efdcc9198862 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:48.414831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f86cad-5ec1-4ec4-95f5-581e3892383c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:48.415568 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40cb70-13d3-4fb3-bf05-efdcc9198862 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:48.420675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 454/898] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:48 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:48.421512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f86cad-5ec1-4ec4-95f5-581e3892383c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:48.421886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 445/899] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 22:20:48 2022] GET /identity/v3/endpoints?service_id=d6abd1e0a24443f19918c8a9567fbc9f&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:48.422722 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40cb70-13d3-4fb3-bf05-efdcc9198862 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:48.424197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:48.486886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:48.549986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40cb70-13d3-4fb3-bf05-efdcc9198862 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:48.550205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f86cad-5ec1-4ec4-95f5-581e3892383c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:48.590029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c08f2-ce12-4c74-8278-9a7fc6bf3729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:48.590844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c08f2-ce12-4c74-8278-9a7fc6bf3729 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.591135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c08f2-ce12-4c74-8278-9a7fc6bf3729 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.591242 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e99b4-195e-4bc8-a516-27387b36b5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:48.591440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c08f2-ce12-4c74-8278-9a7fc6bf3729 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.592383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e99b4-195e-4bc8-a516-27387b36b5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.592525 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c08f2-ce12-4c74-8278-9a7fc6bf3729 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:48.592621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e99b4-195e-4bc8-a516-27387b36b5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.592621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e99b4-195e-4bc8-a516-27387b36b5e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.593191 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e99b4-195e-4bc8-a516-27387b36b5e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:48.593497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e99b4-195e-4bc8-a516-27387b36b5e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:48.593600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c08f2-ce12-4c74-8278-9a7fc6bf3729 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:48.594950 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e99b4-195e-4bc8-a516-27387b36b5e1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-df9c08f2-ce12-4c74-8278-9a7fc6bf3729 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:48.606923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:48.608160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:48.609234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 446/900] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:48.610357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c08f2-ce12-4c74-8278-9a7fc6bf3729 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:48.618964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 455/901] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:48.619701 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e99b4-195e-4bc8-a516-27387b36b5e1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:48.695247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:48.732336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c08f2-ce12-4c74-8278-9a7fc6bf3729 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:48.741292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e99b4-195e-4bc8-a516-27387b36b5e1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:48.742103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:48.770945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52fe93-fa62-44e1-9199-b9b0e3207c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:48.771622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52fe93-fa62-44e1-9199-b9b0e3207c20 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.771908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52fe93-fa62-44e1-9199-b9b0e3207c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.772188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52fe93-fa62-44e1-9199-b9b0e3207c20 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.772917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52fe93-fa62-44e1-9199-b9b0e3207c20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:48.774296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52fe93-fa62-44e1-9199-b9b0e3207c20 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:48.784521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 447/902] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 22:20:48 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:20:48.784521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52fe93-fa62-44e1-9199-b9b0e3207c20 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:48.797116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897853b-1604-444e-8aa1-ae89712311ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:48.797825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897853b-1604-444e-8aa1-ae89712311ab None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.798111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897853b-1604-444e-8aa1-ae89712311ab None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.798387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897853b-1604-444e-8aa1-ae89712311ab None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.799130 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897853b-1604-444e-8aa1-ae89712311ab None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:48.799440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897853b-1604-444e-8aa1-ae89712311ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:48.800983 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897853b-1604-444e-8aa1-ae89712311ab None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:48.808432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 456/903] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:48.814486 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897853b-1604-444e-8aa1-ae89712311ab None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:48.930662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52fe93-fa62-44e1-9199-b9b0e3207c20 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:48.971896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897853b-1604-444e-8aa1-ae89712311ab None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:48.972860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b475b0-94bd-45b2-9360-5d9e75890cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:48.973674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b475b0-94bd-45b2-9360-5d9e75890cce None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:48.973960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b475b0-94bd-45b2-9360-5d9e75890cce None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:48.974244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b475b0-94bd-45b2-9360-5d9e75890cce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:48.982779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b475b0-94bd-45b2-9360-5d9e75890cce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:48.984247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b475b0-94bd-45b2-9360-5d9e75890cce None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-67b475b0-94bd-45b2-9360-5d9e75890cce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:49.000806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:49.002590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:49.004232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:49.004232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:49.004232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:49.004232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:49.004232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:49.004232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:49.004232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:49.004232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:49.004232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 448/904] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:49.008468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b475b0-94bd-45b2-9360-5d9e75890cce None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:49.028870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:49.032368 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a2f2-c34c-4166-bdad-43c5916a8f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:49.033803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a2f2-c34c-4166-bdad-43c5916a8f67 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:49.034640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a2f2-c34c-4166-bdad-43c5916a8f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:49.035572 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a2f2-c34c-4166-bdad-43c5916a8f67 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:49.037166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a2f2-c34c-4166-bdad-43c5916a8f67 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:49.037995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a2f2-c34c-4166-bdad-43c5916a8f67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:49.041086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a2f2-c34c-4166-bdad-43c5916a8f67 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:49.051525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 457/905] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:49.053052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:49.062353 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a2f2-c34c-4166-bdad-43c5916a8f67 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:49.162455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b475b0-94bd-45b2-9360-5d9e75890cce None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:49.213981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a2f2-c34c-4166-bdad-43c5916a8f67 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:49.218157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2a6e7-35d0-4497-ba9a-91bd08babdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:49.219352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2a6e7-35d0-4497-ba9a-91bd08babdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:49.219899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2a6e7-35d0-4497-ba9a-91bd08babdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:49.220414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2a6e7-35d0-4497-ba9a-91bd08babdb4 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:49.221856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2a6e7-35d0-4497-ba9a-91bd08babdb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:49.222438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2a6e7-35d0-4497-ba9a-91bd08babdb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:49.224879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2a6e7-35d0-4497-ba9a-91bd08babdb4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:49.247376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 449/906] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:49.253399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2a6e7-35d0-4497-ba9a-91bd08babdb4 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:49.271309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5f9c1-15ba-4f5b-8adb-1bc9dccaefe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:49.273534 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5f9c1-15ba-4f5b-8adb-1bc9dccaefe1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:49.274034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5f9c1-15ba-4f5b-8adb-1bc9dccaefe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:49.274523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5f9c1-15ba-4f5b-8adb-1bc9dccaefe1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:49.277658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5f9c1-15ba-4f5b-8adb-1bc9dccaefe1 None admin] RBAC: Adding query filter params (interface=public, service_id=0c38f8726aa54776b2f382955b9e2eee, region_id=RegionOne) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:49.278183 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5f9c1-15ba-4f5b-8adb-1bc9dccaefe1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:49.280471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5f9c1-15ba-4f5b-8adb-1bc9dccaefe1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:49.288991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 458/907] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 22:20:49 2022] GET /identity/v3/endpoints?service_id=0c38f8726aa54776b2f382955b9e2eee&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:49.295324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5f9c1-15ba-4f5b-8adb-1bc9dccaefe1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:49.363185 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:49.382210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:49.393311 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2a6e7-35d0-4497-ba9a-91bd08babdb4 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:49.422804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5f9c1-15ba-4f5b-8adb-1bc9dccaefe1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:49.440202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e54b9c-b0c6-4489-b7f4-b2fa3fd4396d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:49.441424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e54b9c-b0c6-4489-b7f4-b2fa3fd4396d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:49.441854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e54b9c-b0c6-4489-b7f4-b2fa3fd4396d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:49.442291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e54b9c-b0c6-4489-b7f4-b2fa3fd4396d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:49.443375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e54b9c-b0c6-4489-b7f4-b2fa3fd4396d None admin] RBAC: Adding query filter params (role.id=803aed498ea04514b286c6e64ee985cd, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=d53f6748398d45358f7b2d75ec273dd6) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:49.443888 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e54b9c-b0c6-4489-b7f4-b2fa3fd4396d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:49.445522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e54b9c-b0c6-4489-b7f4-b2fa3fd4396d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:49.452031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 450/908] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:49 2022] GET /identity/v3/role_assignments?user.id=d53f6748398d45358f7b2d75ec273dd6&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=803aed498ea04514b286c6e64ee985cd => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:49.464628 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874f0b3-00ee-4944-a94a-c5204074a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:49.465332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874f0b3-00ee-4944-a94a-c5204074a060 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:49.465673 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874f0b3-00ee-4944-a94a-c5204074a060 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:49.465958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874f0b3-00ee-4944-a94a-c5204074a060 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:49.466670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874f0b3-00ee-4944-a94a-c5204074a060 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:49.468129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874f0b3-00ee-4944-a94a-c5204074a060 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:49.475487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 459/909] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 22:20:49 2022] GET /identity/v3/services => generated 1268 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:20:50.542941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39a1fb-278d-49ba-9830-dd40398911ca None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:50.543497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39a1fb-278d-49ba-9830-dd40398911ca None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:50.543497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39a1fb-278d-49ba-9830-dd40398911ca None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:50.545021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 451/910] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:50.550629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56148f7-5f29-46eb-a241-4746f7485f0e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:50.551486 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56148f7-5f29-46eb-a241-4746f7485f0e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:50.552002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56148f7-5f29-46eb-a241-4746f7485f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:50.569743 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a56148f7-5f29-46eb-a241-4746f7485f0e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:50.571298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a56148f7-5f29-46eb-a241-4746f7485f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:50.643848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 460/911] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:50.667784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39a1fb-278d-49ba-9830-dd40398911ca None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:50.777683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39a1fb-278d-49ba-9830-dd40398911ca None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:50.812152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ca49a-08e6-409a-b772-cf33273ac22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:50.812859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ca49a-08e6-409a-b772-cf33273ac22c None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:50.813147 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ca49a-08e6-409a-b772-cf33273ac22c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:50.813427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ca49a-08e6-409a-b772-cf33273ac22c None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:50.814170 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ca49a-08e6-409a-b772-cf33273ac22c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:50.815655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ca49a-08e6-409a-b772-cf33273ac22c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:50.832034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 452/912] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:20:50 2022] POST /identity/v3/services => generated 241 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:51.006404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf8f48-534f-46a3-b81d-9c540d09f2c8 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.006774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf8f48-534f-46a3-b81d-9c540d09f2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.006980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf8f48-534f-46a3-b81d-9c540d09f2c8 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.008522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 461/913] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:51.016519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ba7eb-b31c-4935-9dc7-c806e35d9563 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.016519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ba7eb-b31c-4935-9dc7-c806e35d9563 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.016519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ba7eb-b31c-4935-9dc7-c806e35d9563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.034458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-907ba7eb-b31c-4935-9dc7-c806e35d9563 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:51.036378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-907ba7eb-b31c-4935-9dc7-c806e35d9563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:51.048024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:51.114327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 453/914] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:51.137751 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf8f48-534f-46a3-b81d-9c540d09f2c8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:51.190215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:51.246028 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf8f48-534f-46a3-b81d-9c540d09f2c8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:51.283792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3bc90-f9e7-4dfa-af28-fd0587836097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:51.284702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3bc90-f9e7-4dfa-af28-fd0587836097 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.284930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3bc90-f9e7-4dfa-af28-fd0587836097 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.285207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3bc90-f9e7-4dfa-af28-fd0587836097 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.285875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3bc90-f9e7-4dfa-af28-fd0587836097 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:51.287266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc3bc90-f9e7-4dfa-af28-fd0587836097 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-7bc3bc90-f9e7-4dfa-af28-fd0587836097 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:51.300831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:51.301846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:51.302706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 462/915] 10.222.0.42 () {56 vars in 1132 bytes} [Thu Jul 21 22:20:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:51.310688 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ba7eb-b31c-4935-9dc7-c806e35d9563 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:51.425071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ba7eb-b31c-4935-9dc7-c806e35d9563 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:51.462282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a492b-7492-4d5d-8bb3-6b55af7ebb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:51.462960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a492b-7492-4d5d-8bb3-6b55af7ebb91 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.463242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a492b-7492-4d5d-8bb3-6b55af7ebb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.463511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a492b-7492-4d5d-8bb3-6b55af7ebb91 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.464228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a492b-7492-4d5d-8bb3-6b55af7ebb91 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:51.464555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a492b-7492-4d5d-8bb3-6b55af7ebb91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:51.465953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a492b-7492-4d5d-8bb3-6b55af7ebb91 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:51.471321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 454/916] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:51.476927 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3bc90-f9e7-4dfa-af28-fd0587836097 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:51.596731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3bc90-f9e7-4dfa-af28-fd0587836097 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:51.612912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:51.634878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bd049-58c7-420c-9afe-631573cbb780 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.636057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bd049-58c7-420c-9afe-631573cbb780 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.636826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bd049-58c7-420c-9afe-631573cbb780 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.642637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcdfbf-9f1a-4e45-8ad0-337866b36cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:51.642637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcdfbf-9f1a-4e45-8ad0-337866b36cce None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.642637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcdfbf-9f1a-4e45-8ad0-337866b36cce None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.642637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcdfbf-9f1a-4e45-8ad0-337866b36cce None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.643425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcdfbf-9f1a-4e45-8ad0-337866b36cce None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:51.643635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 455/917] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:51.643735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcdfbf-9f1a-4e45-8ad0-337866b36cce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:51.645434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcdfbf-9f1a-4e45-8ad0-337866b36cce None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:51.645760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a6340-438a-4d0e-91f7-5e61f0ed3cdd None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.646242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a6340-438a-4d0e-91f7-5e61f0ed3cdd None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.646731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a6340-438a-4d0e-91f7-5e61f0ed3cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.652221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 463/918] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:51.658810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcdfbf-9f1a-4e45-8ad0-337866b36cce None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:51.662949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:51.666066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-bd1a6340-438a-4d0e-91f7-5e61f0ed3cdd None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:51.667629 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bd1a6340-438a-4d0e-91f7-5e61f0ed3cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:51.747361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 456/919] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:51.748767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c3d7e-8325-434d-ada1-f1db7a3a17ee None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.749061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c3d7e-8325-434d-ada1-f1db7a3a17ee None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.749342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c3d7e-8325-434d-ada1-f1db7a3a17ee None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.754593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 457/920] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:51.755977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecd39d-4d59-4f6b-9de2-2a4154c4c366 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.756263 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecd39d-4d59-4f6b-9de2-2a4154c4c366 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.756581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecd39d-4d59-4f6b-9de2-2a4154c4c366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.775675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-13ecd39d-4d59-4f6b-9de2-2a4154c4c366 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:51.776335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-13ecd39d-4d59-4f6b-9de2-2a4154c4c366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:51.817190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcdfbf-9f1a-4e45-8ad0-337866b36cce None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:51.854265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a80c5-a69c-4a0a-b233-262d884f71b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:51.855077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a80c5-a69c-4a0a-b233-262d884f71b9 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:51.855328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a80c5-a69c-4a0a-b233-262d884f71b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:51.855647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a80c5-a69c-4a0a-b233-262d884f71b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:51.856344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a80c5-a69c-4a0a-b233-262d884f71b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:51.857149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 458/921] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:51.857760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a80c5-a69c-4a0a-b233-262d884f71b9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:51.861800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecd39d-4d59-4f6b-9de2-2a4154c4c366 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:51.880656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 464/922] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 22:20:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:51.881739 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a80c5-a69c-4a0a-b233-262d884f71b9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:51.942479 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:51.955510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:52.004008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecd39d-4d59-4f6b-9de2-2a4154c4c366 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:52.051138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a80c5-a69c-4a0a-b233-262d884f71b9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:52.056376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e6537-10a9-413f-be82-5b22147764e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:52.057073 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e6537-10a9-413f-be82-5b22147764e5 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:52.057361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e6537-10a9-413f-be82-5b22147764e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:52.057640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e6537-10a9-413f-be82-5b22147764e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:52.071226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e6537-10a9-413f-be82-5b22147764e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-5c1e6537-10a9-413f-be82-5b22147764e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:52.080823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:52.081872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:52.083080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 459/923] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jul 21 22:20:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:52.089462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e6537-10a9-413f-be82-5b22147764e5 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:52.099204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06603d1-49b2-4d6d-8e1a-ea43b10bd766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:52.101075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06603d1-49b2-4d6d-8e1a-ea43b10bd766 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:52.101619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06603d1-49b2-4d6d-8e1a-ea43b10bd766 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:52.101892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06603d1-49b2-4d6d-8e1a-ea43b10bd766 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:52.103522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06603d1-49b2-4d6d-8e1a-ea43b10bd766 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:52.105029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06603d1-49b2-4d6d-8e1a-ea43b10bd766 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-a06603d1-49b2-4d6d-8e1a-ea43b10bd766 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:52.123480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:52.124620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:52.125504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 465/924] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:52.130911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06603d1-49b2-4d6d-8e1a-ea43b10bd766 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:52.218602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e6537-10a9-413f-be82-5b22147764e5 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:52.238506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:52.246399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:52.249576 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06603d1-49b2-4d6d-8e1a-ea43b10bd766 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:52.259143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfa999-6564-4976-94dc-90416258871f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:52.259861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfa999-6564-4976-94dc-90416258871f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:52.260143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfa999-6564-4976-94dc-90416258871f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:52.260419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfa999-6564-4976-94dc-90416258871f None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:52.261219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfa999-6564-4976-94dc-90416258871f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:52.261525 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfa999-6564-4976-94dc-90416258871f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:52.263027 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfa999-6564-4976-94dc-90416258871f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:52.277053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 460/925] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 22:20:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:52.279575 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfa999-6564-4976-94dc-90416258871f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:52.293392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e206d-e091-484e-85a1-bb83268f9a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:52.294445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e206d-e091-484e-85a1-bb83268f9a40 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:52.294954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e206d-e091-484e-85a1-bb83268f9a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:52.295430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e206d-e091-484e-85a1-bb83268f9a40 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:52.296360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e206d-e091-484e-85a1-bb83268f9a40 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:52.296895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e206d-e091-484e-85a1-bb83268f9a40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:52.298486 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e206d-e091-484e-85a1-bb83268f9a40 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:52.304627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 466/926] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:52.309810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e206d-e091-484e-85a1-bb83268f9a40 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:52.416979 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfa999-6564-4976-94dc-90416258871f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:52.438813 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e206d-e091-484e-85a1-bb83268f9a40 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:52.463884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf492ea-11bf-4b9d-83bf-2c7d7751f0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:52.464880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf492ea-11bf-4b9d-83bf-2c7d7751f0ca None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:52.465763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf492ea-11bf-4b9d-83bf-2c7d7751f0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:52.466212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf492ea-11bf-4b9d-83bf-2c7d7751f0ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:52.481483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf492ea-11bf-4b9d-83bf-2c7d7751f0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:52.483223 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf492ea-11bf-4b9d-83bf-2c7d7751f0ca None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:52.484402 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d504cd2-008f-4d45-910d-a795219978f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:52.485626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d504cd2-008f-4d45-910d-a795219978f2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:52.485978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d504cd2-008f-4d45-910d-a795219978f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:52.486330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d504cd2-008f-4d45-910d-a795219978f2 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:52.487657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d504cd2-008f-4d45-910d-a795219978f2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:52.488038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d504cd2-008f-4d45-910d-a795219978f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:52.489975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d504cd2-008f-4d45-910d-a795219978f2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:52.498383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 467/927] 10.222.0.42 () {56 vars in 1114 bytes} [Thu Jul 21 22:20:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:52.503774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d504cd2-008f-4d45-910d-a795219978f2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-ccf492ea-11bf-4b9d-83bf-2c7d7751f0ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:52.504875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:52.505746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:52.506631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:52.506631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:52.506631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:52.506631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:52.510093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 461/928] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:52.514020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf492ea-11bf-4b9d-83bf-2c7d7751f0ca None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:52.537312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:52.574062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:52.645705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d504cd2-008f-4d45-910d-a795219978f2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:52.664707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf492ea-11bf-4b9d-83bf-2c7d7751f0ca None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:52.696194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11054c35-bac8-41ea-8775-93ddb0ae10e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:52.698052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11054c35-bac8-41ea-8775-93ddb0ae10e5 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:52.698578 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11054c35-bac8-41ea-8775-93ddb0ae10e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:52.699181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11054c35-bac8-41ea-8775-93ddb0ae10e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:52.700535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11054c35-bac8-41ea-8775-93ddb0ae10e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:52.702506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11054c35-bac8-41ea-8775-93ddb0ae10e5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:52.705200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c572e4f-05d6-4c30-b70f-94edfec0f884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:52.706155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c572e4f-05d6-4c30-b70f-94edfec0f884 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:52.706647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c572e4f-05d6-4c30-b70f-94edfec0f884 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:52.707119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c572e4f-05d6-4c30-b70f-94edfec0f884 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:52.708053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c572e4f-05d6-4c30-b70f-94edfec0f884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:52.708600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c572e4f-05d6-4c30-b70f-94edfec0f884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:52.710348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c572e4f-05d6-4c30-b70f-94edfec0f884 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:52.725866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 462/929] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:52.736590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 468/930] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 22:20:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:52.738993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c572e4f-05d6-4c30-b70f-94edfec0f884 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:52.851267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:52.859276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c572e4f-05d6-4c30-b70f-94edfec0f884 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:52.904437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdc70b-93a9-4bed-af32-01f7e4545f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:52.905301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdc70b-93a9-4bed-af32-01f7e4545f7e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:52.905587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdc70b-93a9-4bed-af32-01f7e4545f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:52.905863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdc70b-93a9-4bed-af32-01f7e4545f7e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:52.914320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdc70b-93a9-4bed-af32-01f7e4545f7e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:52.916063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdc70b-93a9-4bed-af32-01f7e4545f7e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-b3cdc70b-93a9-4bed-af32-01f7e4545f7e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:52.932418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:52.933453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:52.934407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:52.934407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:52.934407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:52.934407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:52.934407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:52.934407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:52.934407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:52.934407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:52.934407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:52.934407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:52.934407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 463/931] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 22:20:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:52.940762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11054c35-bac8-41ea-8775-93ddb0ae10e5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:53.024053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:53.031563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5a787-7b1a-499f-bf7f-b988d997e89f None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:53.031874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5a787-7b1a-499f-bf7f-b988d997e89f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:53.032134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5a787-7b1a-499f-bf7f-b988d997e89f None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:53.033806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 464/932] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:53.039506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ef2df-1305-4b04-8b65-7f6402ba5c13 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:53.039803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ef2df-1305-4b04-8b65-7f6402ba5c13 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:53.040075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ef2df-1305-4b04-8b65-7f6402ba5c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:53.052387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11054c35-bac8-41ea-8775-93ddb0ae10e5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:53.060569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-016ef2df-1305-4b04-8b65-7f6402ba5c13 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:53.061785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-016ef2df-1305-4b04-8b65-7f6402ba5c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:53.090517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103aca39-d3d5-4b2c-b309-d9dfc9f31395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:53.091195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103aca39-d3d5-4b2c-b309-d9dfc9f31395 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:53.091479 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103aca39-d3d5-4b2c-b309-d9dfc9f31395 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:53.091759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103aca39-d3d5-4b2c-b309-d9dfc9f31395 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:53.092660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103aca39-d3d5-4b2c-b309-d9dfc9f31395 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:53.092964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103aca39-d3d5-4b2c-b309-d9dfc9f31395 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:53.094418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103aca39-d3d5-4b2c-b309-d9dfc9f31395 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:53.119599 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 469/933] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jul 21 22:20:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:53.122197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103aca39-d3d5-4b2c-b309-d9dfc9f31395 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:53.151140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 465/934] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:53.173734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ef2df-1305-4b04-8b65-7f6402ba5c13 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:53.252376 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103aca39-d3d5-4b2c-b309-d9dfc9f31395 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:53.275156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:53.291742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59ea01-415a-4144-9694-891d6217cffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:53.292709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59ea01-415a-4144-9694-891d6217cffe None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:53.293114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59ea01-415a-4144-9694-891d6217cffe None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:53.293387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59ea01-415a-4144-9694-891d6217cffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:53.295656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:53.305705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59ea01-415a-4144-9694-891d6217cffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:53.307331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59ea01-415a-4144-9694-891d6217cffe None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:53.308219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ef2df-1305-4b04-8b65-7f6402ba5c13 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-eb59ea01-415a-4144-9694-891d6217cffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:53.327545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:53.328528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:53.329432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:53.329432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:53.329432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:53.329432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:53.334387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 470/935] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:53.335332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59ea01-415a-4144-9694-891d6217cffe None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:53.355217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd516d-ab7a-4a4c-84b9-86d7563a3b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:53.356786 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd516d-ab7a-4a4c-84b9-86d7563a3b43 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:53.357251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd516d-ab7a-4a4c-84b9-86d7563a3b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:53.357735 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd516d-ab7a-4a4c-84b9-86d7563a3b43 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:53.358623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd516d-ab7a-4a4c-84b9-86d7563a3b43 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:53.361021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd516d-ab7a-4a4c-84b9-86d7563a3b43 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-23fd516d-ab7a-4a4c-84b9-86d7563a3b43 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:53.378360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:53.379448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:53.380296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:53.384641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 466/936] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:53.388291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd516d-ab7a-4a4c-84b9-86d7563a3b43 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:53.451638 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59ea01-415a-4144-9694-891d6217cffe None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:53.490826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6d05-f74d-4340-854f-03f183a6f70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:53.491868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6d05-f74d-4340-854f-03f183a6f70b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:53.492894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6d05-f74d-4340-854f-03f183a6f70b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:53.493445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6d05-f74d-4340-854f-03f183a6f70b None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:53.494393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6d05-f74d-4340-854f-03f183a6f70b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:53.495447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6d05-f74d-4340-854f-03f183a6f70b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:53.497865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6d05-f74d-4340-854f-03f183a6f70b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:53.515163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 471/937] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:53.521129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6d05-f74d-4340-854f-03f183a6f70b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:53.526071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd516d-ab7a-4a4c-84b9-86d7563a3b43 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:53.568874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda7a7d-7bcf-40b9-ae4c-ed04c811f6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:53.570423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda7a7d-7bcf-40b9-ae4c-ed04c811f6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:53.570926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda7a7d-7bcf-40b9-ae4c-ed04c811f6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:53.572598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda7a7d-7bcf-40b9-ae4c-ed04c811f6a0 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:53.574130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda7a7d-7bcf-40b9-ae4c-ed04c811f6a0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:53.574674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda7a7d-7bcf-40b9-ae4c-ed04c811f6a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:53.577848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda7a7d-7bcf-40b9-ae4c-ed04c811f6a0 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:53.578346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:53.588206 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 467/938] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:53.594297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda7a7d-7bcf-40b9-ae4c-ed04c811f6a0 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:53.607832 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:53.659318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6d05-f74d-4340-854f-03f183a6f70b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:53.696241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b988e-1add-4803-980a-8f4c278998d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:53.697912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b988e-1add-4803-980a-8f4c278998d0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:53.698147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b988e-1add-4803-980a-8f4c278998d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:53.698392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b988e-1add-4803-980a-8f4c278998d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:53.709067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b988e-1add-4803-980a-8f4c278998d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:53.711754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b988e-1add-4803-980a-8f4c278998d0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:53.720579 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda7a7d-7bcf-40b9-ae4c-ed04c811f6a0 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-105b988e-1add-4803-980a-8f4c278998d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:53.734966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:53.735947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:53.736911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:53.736911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:53.736911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:53.736911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:53.736911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:53.736911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:53.736911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:53.736911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:53.742804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 472/939] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:53.743971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b988e-1add-4803-980a-8f4c278998d0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:53.763267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0470f-fee9-4c01-b018-58f6bf51e44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:53.763946 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0470f-fee9-4c01-b018-58f6bf51e44a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:53.764288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0470f-fee9-4c01-b018-58f6bf51e44a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:53.764616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0470f-fee9-4c01-b018-58f6bf51e44a None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:53.765397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0470f-fee9-4c01-b018-58f6bf51e44a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:53.765709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0470f-fee9-4c01-b018-58f6bf51e44a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:53.767150 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0470f-fee9-4c01-b018-58f6bf51e44a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:53.778852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 468/940] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:53.779706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0470f-fee9-4c01-b018-58f6bf51e44a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:53.861256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:53.882851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b988e-1add-4803-980a-8f4c278998d0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:53.890226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0470f-fee9-4c01-b018-58f6bf51e44a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:53.894252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:53.925836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90b9b6-9f12-4fd9-ba2b-fb29b2071cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:53.926577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90b9b6-9f12-4fd9-ba2b-fb29b2071cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:53.926860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90b9b6-9f12-4fd9-ba2b-fb29b2071cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:53.927181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90b9b6-9f12-4fd9-ba2b-fb29b2071cf5 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:53.927974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90b9b6-9f12-4fd9-ba2b-fb29b2071cf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:53.928283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90b9b6-9f12-4fd9-ba2b-fb29b2071cf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:53.929880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90b9b6-9f12-4fd9-ba2b-fb29b2071cf5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:53.940092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c9432-9104-43d8-832f-04af5306dc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:53.941252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c9432-9104-43d8-832f-04af5306dc49 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:53.941696 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c9432-9104-43d8-832f-04af5306dc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:53.942144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c9432-9104-43d8-832f-04af5306dc49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:53.943111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c9432-9104-43d8-832f-04af5306dc49 None admin] RBAC: Adding query filter params (interface=public, service_id=6fe82fabd8244b168199d66fdb0f3b4c, region_id=RegionOne) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:53.943567 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c9432-9104-43d8-832f-04af5306dc49 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:53.945130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c9432-9104-43d8-832f-04af5306dc49 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:53.947882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 473/941] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:53.949887 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90b9b6-9f12-4fd9-ba2b-fb29b2071cf5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:53.952897 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 469/942] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 22:20:53 2022] GET /identity/v3/endpoints?service_id=6fe82fabd8244b168199d66fdb0f3b4c&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:53.959794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c9432-9104-43d8-832f-04af5306dc49 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:54.078470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90b9b6-9f12-4fd9-ba2b-fb29b2071cf5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:54.104154 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c9432-9104-43d8-832f-04af5306dc49 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:54.118029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca8e3c-62e6-4cd4-a83d-63d94d699425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:54.118974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca8e3c-62e6-4cd4-a83d-63d94d699425 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:54.119259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca8e3c-62e6-4cd4-a83d-63d94d699425 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:54.119621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca8e3c-62e6-4cd4-a83d-63d94d699425 None admin] PATH_INFO: `/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/d53f6748398d45358f7b2d75ec273dd6/roles/803aed498ea04514b286c6e64ee985cd` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:54.121942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca8e3c-62e6-4cd4-a83d-63d94d699425 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0638fe2ce7a40299f2d654f58ed535c, user_id=d53f6748398d45358f7b2d75ec273dd6, role_id=803aed498ea04514b286c6e64ee985cd)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:54.123383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca8e3c-62e6-4cd4-a83d-63d94d699425 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:54.137334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 474/943] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:20:53 2022] PUT /identity/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/d53f6748398d45358f7b2d75ec273dd6/roles/803aed498ea04514b286c6e64ee985cd => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:20:54.139036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7873-2814-4181-8fad-0fab65e571b6 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:54.139328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7873-2814-4181-8fad-0fab65e571b6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:54.139606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7873-2814-4181-8fad-0fab65e571b6 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:54.141103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 475/944] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:54.147057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6295b44-f873-4a27-877e-5ad5a52fda07 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:54.147333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6295b44-f873-4a27-877e-5ad5a52fda07 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:54.147564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6295b44-f873-4a27-877e-5ad5a52fda07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:54.152238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da079854-54ab-4470-8b47-ad27c411be3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:54.153415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da079854-54ab-4470-8b47-ad27c411be3e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:54.154189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da079854-54ab-4470-8b47-ad27c411be3e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:54.155734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da079854-54ab-4470-8b47-ad27c411be3e None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:54.156914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da079854-54ab-4470-8b47-ad27c411be3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:54.158615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da079854-54ab-4470-8b47-ad27c411be3e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:54.166934 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b6295b44-f873-4a27-877e-5ad5a52fda07 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:54.176544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b6295b44-f873-4a27-877e-5ad5a52fda07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:54.176767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 470/945] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 22:20:53 2022] GET /identity/v3/services => generated 1497 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:20:54.206754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:54.270868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 476/946] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:54.291293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da079854-54ab-4470-8b47-ad27c411be3e None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:54.294583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:54.412117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da079854-54ab-4470-8b47-ad27c411be3e None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:54.459882 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a9736-890a-471a-a141-23deedbb4448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:54.460726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a9736-890a-471a-a141-23deedbb4448 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:54.461010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a9736-890a-471a-a141-23deedbb4448 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:54.461286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a9736-890a-471a-a141-23deedbb4448 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:54.461950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a9736-890a-471a-a141-23deedbb4448 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:54.463766 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a9736-890a-471a-a141-23deedbb4448 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-dd7a9736-890a-471a-a141-23deedbb4448 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:54.476098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:54.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:20:54.478171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:54.482618 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 471/947] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:54.483987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6295b44-f873-4a27-877e-5ad5a52fda07 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:54.588095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6295b44-f873-4a27-877e-5ad5a52fda07 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:54.627406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2453f-6aa4-4290-9567-6e507417c28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:54.628501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2453f-6aa4-4290-9567-6e507417c28d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:54.629003 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2453f-6aa4-4290-9567-6e507417c28d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:54.629481 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2453f-6aa4-4290-9567-6e507417c28d None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:54.630410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2453f-6aa4-4290-9567-6e507417c28d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:54.630957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2453f-6aa4-4290-9567-6e507417c28d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:54.632573 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2453f-6aa4-4290-9567-6e507417c28d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:54.639642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 477/948] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:54.645471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a9736-890a-471a-a141-23deedbb4448 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:54.720070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:54.763494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a9736-890a-471a-a141-23deedbb4448 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:54.800570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7769bea-a52d-4dce-bfb9-c48665d32c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:54.801247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7769bea-a52d-4dce-bfb9-c48665d32c7e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:54.801526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7769bea-a52d-4dce-bfb9-c48665d32c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:54.801800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7769bea-a52d-4dce-bfb9-c48665d32c7e None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:54.802514 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7769bea-a52d-4dce-bfb9-c48665d32c7e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:54.802820 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7769bea-a52d-4dce-bfb9-c48665d32c7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:54.804258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7769bea-a52d-4dce-bfb9-c48665d32c7e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:54.815338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 472/949] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:55.025235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82796036-0bee-45bc-aeda-b7760fe4ad52 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:55.025745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82796036-0bee-45bc-aeda-b7760fe4ad52 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:55.025745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82796036-0bee-45bc-aeda-b7760fe4ad52 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:55.027290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 478/950] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:55.033566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bbc06-aa74-45c3-8ccf-eb4a5569ccf0 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:55.033855 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bbc06-aa74-45c3-8ccf-eb4a5569ccf0 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:55.034124 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bbc06-aa74-45c3-8ccf-eb4a5569ccf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:55.052264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ed6bbc06-aa74-45c3-8ccf-eb4a5569ccf0 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:55.053713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ed6bbc06-aa74-45c3-8ccf-eb4a5569ccf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:55.139248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 473/951] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:55.166659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82796036-0bee-45bc-aeda-b7760fe4ad52 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:55.193757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:55.282290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82796036-0bee-45bc-aeda-b7760fe4ad52 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:55.319230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33a2b8-f3f7-420c-813b-05a671aab969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:55.320307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33a2b8-f3f7-420c-813b-05a671aab969 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:55.320815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33a2b8-f3f7-420c-813b-05a671aab969 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:55.321334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33a2b8-f3f7-420c-813b-05a671aab969 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:55.322938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33a2b8-f3f7-420c-813b-05a671aab969 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:55.324625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33a2b8-f3f7-420c-813b-05a671aab969 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:55.326289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: WARNING keystone.server.flask.common [None req-ec33a2b8-f3f7-420c-813b-05a671aab969 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 22:20:55.346926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.sql.core [None req-ec33a2b8-f3f7-420c-813b-05a671aab969 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 22:20:55.346926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 22:20:55.346926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [parameters: {'id': '7e317d6192db4a69b942bbf53a1e4f54', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 22:20:55.346926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80315) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ec33a2b8-f3f7-420c-813b-05a671aab969 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7e317d6192db4a69b942bbf53a1e4f54.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7e317d6192db4a69b942bbf53a1e4f54. Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:20:55.355698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result.read() Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 22:20:55.356614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application yield resource Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise self.value Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application session.commit() Jul 21 22:20:55.357781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.session.flush() Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application compat.raise_( Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 22:20:55.358938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 22:20:55.360045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result.read() Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:20:55.360870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [parameters: {'id': '7e317d6192db4a69b942bbf53a1e4f54', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:55.361734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:55.362517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:55.363648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:55.363648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 22:20:55.363648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 22:20:55.363648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:55.363648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:55.363648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 22:20:55.363648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 22:20:55.363648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7e317d6192db4a69b942bbf53a1e4f54. Jul 21 22:20:55.363648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:55.366141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 479/952] 10.222.0.42 () {60 vars in 1140 bytes} [Thu Jul 21 22:20:55 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:55.372771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6bbc06-aa74-45c3-8ccf-eb4a5569ccf0 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:55.448573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:55.497108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6bbc06-aa74-45c3-8ccf-eb4a5569ccf0 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:55.544321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de59d1-e855-4766-a7c8-8669eff34af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:55.545560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de59d1-e855-4766-a7c8-8669eff34af4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:55.545990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de59d1-e855-4766-a7c8-8669eff34af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:55.546421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de59d1-e855-4766-a7c8-8669eff34af4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:55.558350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de59d1-e855-4766-a7c8-8669eff34af4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:55.560784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de59d1-e855-4766-a7c8-8669eff34af4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-c9de59d1-e855-4766-a7c8-8669eff34af4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:55.586794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:55.588551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:55.589495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:55.589495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:55.589495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:55.589495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:55.589495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:55.589495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:55.589495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:55.589495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:55.592577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 474/953] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:55.596125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33a2b8-f3f7-420c-813b-05a671aab969 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:55.690591 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:55.716384 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33a2b8-f3f7-420c-813b-05a671aab969 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:55.752089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2276fc0-affc-4204-b099-d30e9cc7044e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:55.752785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2276fc0-affc-4204-b099-d30e9cc7044e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:55.753128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2276fc0-affc-4204-b099-d30e9cc7044e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:55.753403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2276fc0-affc-4204-b099-d30e9cc7044e None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:55.754209 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2276fc0-affc-4204-b099-d30e9cc7044e None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:55.754520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2276fc0-affc-4204-b099-d30e9cc7044e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:55.756162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2276fc0-affc-4204-b099-d30e9cc7044e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:55.778670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 480/954] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:20:55 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:56.357628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369f1e6-096c-4224-9e1d-83f33376a477 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:56.358527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369f1e6-096c-4224-9e1d-83f33376a477 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:56.358957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369f1e6-096c-4224-9e1d-83f33376a477 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:56.360750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 475/955] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:56.366593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f59fc-2b0d-4710-8c25-b64cfce5c1c1 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:56.367071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f59fc-2b0d-4710-8c25-b64cfce5c1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:56.367165 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f59fc-2b0d-4710-8c25-b64cfce5c1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:56.385475 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-786f59fc-2b0d-4710-8c25-b64cfce5c1c1 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:56.387123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-786f59fc-2b0d-4710-8c25-b64cfce5c1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:56.457402 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 481/956] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:56.475312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e9a9e-eeb7-463d-b0f4-e6d8c5043d16 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:56.475607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e9a9e-eeb7-463d-b0f4-e6d8c5043d16 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:56.475883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e9a9e-eeb7-463d-b0f4-e6d8c5043d16 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:56.477620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 476/957] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:56.482416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f59fc-2b0d-4710-8c25-b64cfce5c1c1 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:56.483224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12a99d-ec36-499b-94f8-21224b93732b None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:56.483514 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12a99d-ec36-499b-94f8-21224b93732b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:56.483791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12a99d-ec36-499b-94f8-21224b93732b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:56.504149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-8d12a99d-ec36-499b-94f8-21224b93732b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:56.505899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8d12a99d-ec36-499b-94f8-21224b93732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:56.563076 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:56.599397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 477/958] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:56.606294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f59fc-2b0d-4710-8c25-b64cfce5c1c1 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:56.626162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12a99d-ec36-499b-94f8-21224b93732b None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:56.647696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c9c4b-9e0c-461f-a485-8c2c8c5af504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:56.648689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c9c4b-9e0c-461f-a485-8c2c8c5af504 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:56.648986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c9c4b-9e0c-461f-a485-8c2c8c5af504 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:56.649266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c9c4b-9e0c-461f-a485-8c2c8c5af504 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:56.666078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c9c4b-9e0c-461f-a485-8c2c8c5af504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-0c3c9c4b-9e0c-461f-a485-8c2c8c5af504 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:20:56.674416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:56.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:56.678765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 482/959] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jul 21 22:20:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:56.688519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c9c4b-9e0c-461f-a485-8c2c8c5af504 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:56.701260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:56.755170 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12a99d-ec36-499b-94f8-21224b93732b None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:56.793008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677c44d-081c-4769-b4a3-12c753bd667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:56.794040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677c44d-081c-4769-b4a3-12c753bd667c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:56.794764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677c44d-081c-4769-b4a3-12c753bd667c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:56.794764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677c44d-081c-4769-b4a3-12c753bd667c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:56.795582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677c44d-081c-4769-b4a3-12c753bd667c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:56.797091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677c44d-081c-4769-b4a3-12c753bd667c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-4677c44d-081c-4769-b4a3-12c753bd667c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:56.810617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:56.811960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:20:56.813079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:56.815105 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 478/960] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:56.824464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677c44d-081c-4769-b4a3-12c753bd667c None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:56.829913 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c9c4b-9e0c-461f-a485-8c2c8c5af504 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:56.861665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:56.871861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19757b2c-6631-4159-9a72-8a0488a4655f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:56.873024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19757b2c-6631-4159-9a72-8a0488a4655f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:56.873532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19757b2c-6631-4159-9a72-8a0488a4655f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:56.874032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19757b2c-6631-4159-9a72-8a0488a4655f None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:56.874984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19757b2c-6631-4159-9a72-8a0488a4655f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:56.875511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19757b2c-6631-4159-9a72-8a0488a4655f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:56.877197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19757b2c-6631-4159-9a72-8a0488a4655f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:56.892128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 483/961] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 22:20:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:56.893998 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19757b2c-6631-4159-9a72-8a0488a4655f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:56.941263 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677c44d-081c-4769-b4a3-12c753bd667c None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:56.956517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:56.979809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020f4c5-1402-4001-b10c-2d6acea61c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:56.980353 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020f4c5-1402-4001-b10c-2d6acea61c8f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:56.980670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020f4c5-1402-4001-b10c-2d6acea61c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:56.980946 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020f4c5-1402-4001-b10c-2d6acea61c8f None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:56.981732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020f4c5-1402-4001-b10c-2d6acea61c8f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:56.982041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020f4c5-1402-4001-b10c-2d6acea61c8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:56.983628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020f4c5-1402-4001-b10c-2d6acea61c8f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:56.992822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 479/962] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:56.994927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020f4c5-1402-4001-b10c-2d6acea61c8f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:57.009303 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19757b2c-6631-4159-9a72-8a0488a4655f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:57.046672 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2557380-dde9-4a89-b35d-dc12bced8749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:57.047868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2557380-dde9-4a89-b35d-dc12bced8749 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.048156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2557380-dde9-4a89-b35d-dc12bced8749 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.048434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2557380-dde9-4a89-b35d-dc12bced8749 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.060329 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2557380-dde9-4a89-b35d-dc12bced8749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:57.061834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2557380-dde9-4a89-b35d-dc12bced8749 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-c2557380-dde9-4a89-b35d-dc12bced8749 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:57.081834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:57.082852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:57.084399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:57.084399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:57.084399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:57.084399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:57.084399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 484/963] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:57.084745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defce32-8927-44c3-a169-03109ddb2ae3 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.084961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defce32-8927-44c3-a169-03109ddb2ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.085205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defce32-8927-44c3-a169-03109ddb2ae3 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.086571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 485/964] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:57.089069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defce32-8927-44c3-a169-03109ddb2ae3 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:57.119078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020f4c5-1402-4001-b10c-2d6acea61c8f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:57.138665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:57.157132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6d376-aa84-4050-93fb-56db9ab77e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:57.157906 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6d376-aa84-4050-93fb-56db9ab77e85 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.158088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6d376-aa84-4050-93fb-56db9ab77e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.158364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6d376-aa84-4050-93fb-56db9ab77e85 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.159072 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6d376-aa84-4050-93fb-56db9ab77e85 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:57.159376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6d376-aa84-4050-93fb-56db9ab77e85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:57.160803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6d376-aa84-4050-93fb-56db9ab77e85 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:57.170669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 480/965] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 22:20:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:57.170669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70fa90-6bd7-4d4c-8c17-58120f0800d8 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.170669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70fa90-6bd7-4d4c-8c17-58120f0800d8 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.170669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70fa90-6bd7-4d4c-8c17-58120f0800d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.184889 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-6b70fa90-6bd7-4d4c-8c17-58120f0800d8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:57.186100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6b70fa90-6bd7-4d4c-8c17-58120f0800d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:57.210694 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defce32-8927-44c3-a169-03109ddb2ae3 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:57.225540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:57.249690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716b133-fb35-4ca7-a46e-68792572ec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:57.250358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716b133-fb35-4ca7-a46e-68792572ec79 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.250644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716b133-fb35-4ca7-a46e-68792572ec79 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.250924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716b133-fb35-4ca7-a46e-68792572ec79 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.251645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716b133-fb35-4ca7-a46e-68792572ec79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:57.251955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716b133-fb35-4ca7-a46e-68792572ec79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:57.253365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716b133-fb35-4ca7-a46e-68792572ec79 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:57.264553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 481/966] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:57.264553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70fa90-6bd7-4d4c-8c17-58120f0800d8 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:57.270374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 486/967] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:57.275840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716b133-fb35-4ca7-a46e-68792572ec79 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:57.375922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70fa90-6bd7-4d4c-8c17-58120f0800d8 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:57.379818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:57.383648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716b133-fb35-4ca7-a46e-68792572ec79 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:57.414493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06384c59-532c-4d57-b8b5-9b520226af66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:57.415294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06384c59-532c-4d57-b8b5-9b520226af66 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.415620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06384c59-532c-4d57-b8b5-9b520226af66 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.415970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06384c59-532c-4d57-b8b5-9b520226af66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.416788 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06384c59-532c-4d57-b8b5-9b520226af66 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:57.418204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06384c59-532c-4d57-b8b5-9b520226af66 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:57.420369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84423b7c-dcb6-4f2e-8a7e-50b99cde708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:57.423817 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84423b7c-dcb6-4f2e-8a7e-50b99cde708d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.423817 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84423b7c-dcb6-4f2e-8a7e-50b99cde708d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.423817 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84423b7c-dcb6-4f2e-8a7e-50b99cde708d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.431806 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84423b7c-dcb6-4f2e-8a7e-50b99cde708d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:57.433327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84423b7c-dcb6-4f2e-8a7e-50b99cde708d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:57.445526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 482/968] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 22:20:57 2022] POST /identity/v3/endpoints => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:57.446680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06384c59-532c-4d57-b8b5-9b520226af66 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-84423b7c-dcb6-4f2e-8a7e-50b99cde708d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:57.449546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:57.450642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:57.451675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:57.451675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:57.451675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:57.451675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:57.451675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:57.451675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:57.451675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:57.451675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:57.451675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:20:57.451675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:57.451675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 487/969] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 22:20:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:57.458515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84423b7c-dcb6-4f2e-8a7e-50b99cde708d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:57.487372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:57.587547 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84423b7c-dcb6-4f2e-8a7e-50b99cde708d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:57.609095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06384c59-532c-4d57-b8b5-9b520226af66 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:57.631206 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819604cf-745d-4446-a5fe-b76f9c6c24cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:57.631954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819604cf-745d-4446-a5fe-b76f9c6c24cd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.632242 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819604cf-745d-4446-a5fe-b76f9c6c24cd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.632544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819604cf-745d-4446-a5fe-b76f9c6c24cd None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.634767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819604cf-745d-4446-a5fe-b76f9c6c24cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:57.635099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819604cf-745d-4446-a5fe-b76f9c6c24cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:57.637117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819604cf-745d-4446-a5fe-b76f9c6c24cd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:57.652117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc305a9-d6ee-49a2-a42d-7955249b543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:57.652918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc305a9-d6ee-49a2-a42d-7955249b543b None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.653219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc305a9-d6ee-49a2-a42d-7955249b543b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.653482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc305a9-d6ee-49a2-a42d-7955249b543b None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.654228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc305a9-d6ee-49a2-a42d-7955249b543b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:57.655646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc305a9-d6ee-49a2-a42d-7955249b543b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:57.663427 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 488/970] 10.222.0.42 () {56 vars in 1149 bytes} [Thu Jul 21 22:20:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:20:57.674305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819604cf-745d-4446-a5fe-b76f9c6c24cd None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:57.675107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 483/971] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:20:57 2022] POST /identity/v3/services => generated 242 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:20:57.700097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:57.813739 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819604cf-745d-4446-a5fe-b76f9c6c24cd None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:57.861091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6610c50-a1c8-49a1-b719-a4f40458f7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:57.861743 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6610c50-a1c8-49a1-b719-a4f40458f7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:57.862024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6610c50-a1c8-49a1-b719-a4f40458f7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:57.862296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6610c50-a1c8-49a1-b719-a4f40458f7f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:57.940672 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6610c50-a1c8-49a1-b719-a4f40458f7f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:57.942055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6610c50-a1c8-49a1-b719-a4f40458f7f3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:57.958164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:58.051837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d958b6-0ce8-4026-a701-5b215e9b5237 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.052138 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d958b6-0ce8-4026-a701-5b215e9b5237 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.052428 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d958b6-0ce8-4026-a701-5b215e9b5237 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.053828 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 484/973] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:58.054063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 489/973] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:20:57 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:20:58.055143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d958b6-0ce8-4026-a701-5b215e9b5237 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:58.060744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dce2fc-9355-4401-ac4b-278585c4e34b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.061341 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dce2fc-9355-4401-ac4b-278585c4e34b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.061803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dce2fc-9355-4401-ac4b-278585c4e34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.080558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b1dce2fc-9355-4401-ac4b-278585c4e34b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:58.082381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b1dce2fc-9355-4401-ac4b-278585c4e34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:58.157464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:58.180208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d958b6-0ce8-4026-a701-5b215e9b5237 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:58.192302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 490/974] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:58.217802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dce2fc-9355-4401-ac4b-278585c4e34b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:58.223596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405b4a0-70b3-4576-91b0-01ac24cefed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:58.224408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405b4a0-70b3-4576-91b0-01ac24cefed6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.224738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405b4a0-70b3-4576-91b0-01ac24cefed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.225017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405b4a0-70b3-4576-91b0-01ac24cefed6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.237303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405b4a0-70b3-4576-91b0-01ac24cefed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:58.238910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405b4a0-70b3-4576-91b0-01ac24cefed6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-7405b4a0-70b3-4576-91b0-01ac24cefed6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:20:58.258612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:58.260404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:58.262002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:20:58.262002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:20:58.262002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:20:58.262002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:58.262002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 485/975] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:20:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:58.269149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405b4a0-70b3-4576-91b0-01ac24cefed6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:58.272943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:58.346881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dce2fc-9355-4401-ac4b-278585c4e34b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:58.385206 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb39ce4-b8c0-4553-83ca-faa15feda9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:58.386604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb39ce4-b8c0-4553-83ca-faa15feda9e2 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.387058 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb39ce4-b8c0-4553-83ca-faa15feda9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.387577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb39ce4-b8c0-4553-83ca-faa15feda9e2 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.388444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb39ce4-b8c0-4553-83ca-faa15feda9e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:58.390081 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb39ce4-b8c0-4553-83ca-faa15feda9e2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:58.402550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405b4a0-70b3-4576-91b0-01ac24cefed6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:58.406971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.sql.core [None req-eeb39ce4-b8c0-4553-83ca-faa15feda9e2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 22:20:58.406971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 22:20:58.406971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [parameters: {'user_id': 'fb685eacb0a746a3879e86c42792361c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 22:20:58.406971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80315) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-eeb39ce4-b8c0-4553-83ca-faa15feda9e2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result.read() Jul 21 22:20:58.415794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:20:58.417007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application yield resource Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise self.value Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application session.commit() Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 22:20:58.418714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.session.flush() Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application compat.raise_( Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 22:20:58.419804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:20:58.421901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result.read() Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:20:58.424192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fb685eacb0a746a3879e86c42792361c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:58.426909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 22:20:58.429516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 22:20:58.439636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 22:20:58.439636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 22:20:58.439636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 22:20:58.439636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 22:20:58.439636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 22:20:58.439636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:58.439636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 491/976] 10.222.0.42 () {60 vars in 1131 bytes} [Thu Jul 21 22:20:58 2022] POST /identity/v3/users => generated 160 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:58.439636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb39ce4-b8c0-4553-83ca-faa15feda9e2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:58.468657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9b911-3794-4dc4-b625-a6c7c7c5ba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:58.469357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9b911-3794-4dc4-b625-a6c7c7c5ba96 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.469641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9b911-3794-4dc4-b625-a6c7c7c5ba96 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.469941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9b911-3794-4dc4-b625-a6c7c7c5ba96 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.470650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9b911-3794-4dc4-b625-a6c7c7c5ba96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:58.470954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9b911-3794-4dc4-b625-a6c7c7c5ba96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:58.472314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9b911-3794-4dc4-b625-a6c7c7c5ba96 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:58.475470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:58.490510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 486/977] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:58.492537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9b911-3794-4dc4-b625-a6c7c7c5ba96 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:58.579312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb39ce4-b8c0-4553-83ca-faa15feda9e2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:58.587935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:58.603957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9b911-3794-4dc4-b625-a6c7c7c5ba96 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:58.623452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767d8be-8631-433c-9a0f-aef3168eaea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:58.624559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767d8be-8631-433c-9a0f-aef3168eaea1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.624989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767d8be-8631-433c-9a0f-aef3168eaea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.625409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767d8be-8631-433c-9a0f-aef3168eaea1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.634828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767d8be-8631-433c-9a0f-aef3168eaea1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:58.636959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767d8be-8631-433c-9a0f-aef3168eaea1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:58.642141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ef2c7-7a21-403e-bc63-cd4527e0a35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:58.642961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ef2c7-7a21-403e-bc63-cd4527e0a35f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.643250 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ef2c7-7a21-403e-bc63-cd4527e0a35f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.643530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ef2c7-7a21-403e-bc63-cd4527e0a35f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.652067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ef2c7-7a21-403e-bc63-cd4527e0a35f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:58.654187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ef2c7-7a21-403e-bc63-cd4527e0a35f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-7767d8be-8631-433c-9a0f-aef3168eaea1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:20:58.656250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:58.657248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:58.658211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:58.658211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:58.658211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:20:58.658211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:20:58.658211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:20:58.658211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:20:58.658211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:20:58.658211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:20:58.658211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:20:58.658211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:20:58.661349 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 492/978] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:20:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:20:58.668213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767d8be-8631-433c-9a0f-aef3168eaea1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-012ef2c7-7a21-403e-bc63-cd4527e0a35f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:20:58.682383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:20:58.683346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:20:58.684264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:20:58.684264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:20:58.684264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:20:58.684264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:20:58.684264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:20:58.684264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:20:58.684264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:20:58.684264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:20:58.688443 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 487/979] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:20:58.691441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ef2c7-7a21-403e-bc63-cd4527e0a35f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:58.760342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:58.806253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767d8be-8631-433c-9a0f-aef3168eaea1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:58.829507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ef2c7-7a21-403e-bc63-cd4527e0a35f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:58.855574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8a970-1455-41d9-b616-3974fcb20a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:58.856272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8a970-1455-41d9-b616-3974fcb20a95 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.856602 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8a970-1455-41d9-b616-3974fcb20a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.856886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8a970-1455-41d9-b616-3974fcb20a95 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.857910 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8a970-1455-41d9-b616-3974fcb20a95 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:58.858229 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8a970-1455-41d9-b616-3974fcb20a95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:58.860086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8a970-1455-41d9-b616-3974fcb20a95 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:58.888884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2867ba-3f34-43ca-9359-bb29e3ea3772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:58.889615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2867ba-3f34-43ca-9359-bb29e3ea3772 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:58.889903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2867ba-3f34-43ca-9359-bb29e3ea3772 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:58.890182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2867ba-3f34-43ca-9359-bb29e3ea3772 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:58.891006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2867ba-3f34-43ca-9359-bb29e3ea3772 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:58.891320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2867ba-3f34-43ca-9359-bb29e3ea3772 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:58.892858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2867ba-3f34-43ca-9359-bb29e3ea3772 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:58.913760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 488/980] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:20:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:58.914262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 493/981] 10.222.0.42 () {56 vars in 1101 bytes} [Thu Jul 21 22:20:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:58.919065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2867ba-3f34-43ca-9359-bb29e3ea3772 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:59.046029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2867ba-3f34-43ca-9359-bb29e3ea3772 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:59.066518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:59.090239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dc24d-4f49-4500-847d-e643085801bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:59.090927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dc24d-4f49-4500-847d-e643085801bb None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:59.091212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dc24d-4f49-4500-847d-e643085801bb None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:59.091493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dc24d-4f49-4500-847d-e643085801bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:59.092342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dc24d-4f49-4500-847d-e643085801bb None admin] RBAC: Adding query filter params (role.id=803aed498ea04514b286c6e64ee985cd, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=d53f6748398d45358f7b2d75ec273dd6) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:20:59.092692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dc24d-4f49-4500-847d-e643085801bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:59.094131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dc24d-4f49-4500-847d-e643085801bb None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:20:59.103952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 489/982] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:20:58 2022] GET /identity/v3/role_assignments?user.id=d53f6748398d45358f7b2d75ec273dd6&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=803aed498ea04514b286c6e64ee985cd => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:20:59.722903 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031af7e-df20-4625-88fd-8a43f7eada4c None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:59.723394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031af7e-df20-4625-88fd-8a43f7eada4c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:59.723394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031af7e-df20-4625-88fd-8a43f7eada4c None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:59.724812 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 494/983] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:20:59.730075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceea130-ae2e-4440-8729-946eedfd98c7 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:59.730460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceea130-ae2e-4440-8729-946eedfd98c7 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:59.730628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceea130-ae2e-4440-8729-946eedfd98c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:59.747979 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-fceea130-ae2e-4440-8729-946eedfd98c7 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:59.749193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fceea130-ae2e-4440-8729-946eedfd98c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:59.818308 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 490/984] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:20:59.842364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031af7e-df20-4625-88fd-8a43f7eada4c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:59.858909 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe1782-7c7d-4a8b-8844-2f21a92ac1d8 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:59.859223 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe1782-7c7d-4a8b-8844-2f21a92ac1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:59.859474 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe1782-7c7d-4a8b-8844-2f21a92ac1d8 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:59.861015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 491/985] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:20:59.866723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900acc00-4d58-446a-aac9-7f54da21d6c9 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:59.867287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900acc00-4d58-446a-aac9-7f54da21d6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:59.867572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900acc00-4d58-446a-aac9-7f54da21d6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:59.869744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:20:59.886099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-900acc00-4d58-446a-aac9-7f54da21d6c9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:20:59.887597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-900acc00-4d58-446a-aac9-7f54da21d6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:20:59.954480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031af7e-df20-4625-88fd-8a43f7eada4c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:20:59.976298 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 492/986] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:20:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:20:59.990791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900acc00-4d58-446a-aac9-7f54da21d6c9 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:20:59.994524 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b030f11-d2b7-4e0d-bcb7-f067b2e61d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:20:59.995342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b030f11-d2b7-4e0d-bcb7-f067b2e61d17 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:20:59.995628 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b030f11-d2b7-4e0d-bcb7-f067b2e61d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:20:59.995908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b030f11-d2b7-4e0d-bcb7-f067b2e61d17 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:20:59.996672 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b030f11-d2b7-4e0d-bcb7-f067b2e61d17 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:20:59.998064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b030f11-d2b7-4e0d-bcb7-f067b2e61d17 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-6b030f11-d2b7-4e0d-bcb7-f067b2e61d17 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:00.011018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:00.012075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:21:00.013016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:00.016810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 495/987] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 22:20:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:00.018295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b030f11-d2b7-4e0d-bcb7-f067b2e61d17 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:00.036671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:00.098823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900acc00-4d58-446a-aac9-7f54da21d6c9 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:00.117565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:00.131056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b030f11-d2b7-4e0d-bcb7-f067b2e61d17 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:00.134921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3813fe-2692-425f-abea-c65fbce073ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:00.135733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3813fe-2692-425f-abea-c65fbce073ee None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:00.136022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3813fe-2692-425f-abea-c65fbce073ee None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:00.136301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3813fe-2692-425f-abea-c65fbce073ee None admin] PATH_INFO: `/v3/services/compute` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:00.137001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3813fe-2692-425f-abea-c65fbce073ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:00.138384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3813fe-2692-425f-abea-c65fbce073ee None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-9b3813fe-2692-425f-abea-c65fbce073ee None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:00.150662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:00.151809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:00.153601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 493/988] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:20:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:00.157857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3813fe-2692-425f-abea-c65fbce073ee None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:00.171384 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589e2eb-16fa-4b61-8542-bb752c8ee6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:00.172049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589e2eb-16fa-4b61-8542-bb752c8ee6ef None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:00.172331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589e2eb-16fa-4b61-8542-bb752c8ee6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:00.172645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589e2eb-16fa-4b61-8542-bb752c8ee6ef None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:00.173349 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589e2eb-16fa-4b61-8542-bb752c8ee6ef None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:00.173653 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589e2eb-16fa-4b61-8542-bb752c8ee6ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:00.175014 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589e2eb-16fa-4b61-8542-bb752c8ee6ef None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:00.180039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 496/989] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 22:21:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:00.186088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589e2eb-16fa-4b61-8542-bb752c8ee6ef None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:00.265994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3813fe-2692-425f-abea-c65fbce073ee None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:00.268752 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:00.295961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589e2eb-16fa-4b61-8542-bb752c8ee6ef None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:00.307747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdad3b-9fed-442c-8e07-461a88cfdef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:00.309286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdad3b-9fed-442c-8e07-461a88cfdef4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:00.309778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdad3b-9fed-442c-8e07-461a88cfdef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:00.310065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdad3b-9fed-442c-8e07-461a88cfdef4 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:00.310821 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdad3b-9fed-442c-8e07-461a88cfdef4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:00.311140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdad3b-9fed-442c-8e07-461a88cfdef4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:00.312571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdad3b-9fed-442c-8e07-461a88cfdef4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:00.317909 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 494/990] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:21:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:00.324056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdad3b-9fed-442c-8e07-461a88cfdef4 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:00.336865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa356f-2025-473a-a657-f6474506829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:00.338094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa356f-2025-473a-a657-f6474506829c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:00.338570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa356f-2025-473a-a657-f6474506829c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:00.339200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa356f-2025-473a-a657-f6474506829c None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:00.340513 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa356f-2025-473a-a657-f6474506829c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:00.341284 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa356f-2025-473a-a657-f6474506829c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:00.343024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa356f-2025-473a-a657-f6474506829c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:00.350173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 497/991] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 22:21:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:00.356425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa356f-2025-473a-a657-f6474506829c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:00.377019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:00.441750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdad3b-9fed-442c-8e07-461a88cfdef4 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:00.470111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa356f-2025-473a-a657-f6474506829c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:00.480362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2f29a-0a3c-4905-8e2d-909ba13a3997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:00.481699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2f29a-0a3c-4905-8e2d-909ba13a3997 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:00.481936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2f29a-0a3c-4905-8e2d-909ba13a3997 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:00.483757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2f29a-0a3c-4905-8e2d-909ba13a3997 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:00.483757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2f29a-0a3c-4905-8e2d-909ba13a3997 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:00.483931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2f29a-0a3c-4905-8e2d-909ba13a3997 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:00.485817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2f29a-0a3c-4905-8e2d-909ba13a3997 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:00.496553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 495/992] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:21:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:00.498166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2f29a-0a3c-4905-8e2d-909ba13a3997 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:00.516307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d8ca5-35d8-4b4b-a0ad-86e7d04ed02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:00.516925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d8ca5-35d8-4b4b-a0ad-86e7d04ed02e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:00.517215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d8ca5-35d8-4b4b-a0ad-86e7d04ed02e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:00.517495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d8ca5-35d8-4b4b-a0ad-86e7d04ed02e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:00.518476 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d8ca5-35d8-4b4b-a0ad-86e7d04ed02e None admin] RBAC: Adding query filter params (interface=admin, service_id=6fe82fabd8244b168199d66fdb0f3b4c, region_id=RegionOne) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:00.518993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d8ca5-35d8-4b4b-a0ad-86e7d04ed02e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:00.520379 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d8ca5-35d8-4b4b-a0ad-86e7d04ed02e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:00.530451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 498/993] 10.222.0.42 () {56 vars in 1249 bytes} [Thu Jul 21 22:21:00 2022] GET /identity/v3/endpoints?service_id=6fe82fabd8244b168199d66fdb0f3b4c&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:00.531805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d8ca5-35d8-4b4b-a0ad-86e7d04ed02e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:00.541661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:00.620163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:00.625160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2f29a-0a3c-4905-8e2d-909ba13a3997 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:00.646129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d8ca5-35d8-4b4b-a0ad-86e7d04ed02e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:00.678086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde17bf3-dca8-4459-ac3b-7d2ed8d9aa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:00.679260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde17bf3-dca8-4459-ac3b-7d2ed8d9aa98 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:00.679785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde17bf3-dca8-4459-ac3b-7d2ed8d9aa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:00.680297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde17bf3-dca8-4459-ac3b-7d2ed8d9aa98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:00.682494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde17bf3-dca8-4459-ac3b-7d2ed8d9aa98 None admin] RBAC: Adding query filter params (interface=public, service_id=5e59d451e9ff4295856d2ab0fb976d06, region_id=RegionOne) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:00.683039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde17bf3-dca8-4459-ac3b-7d2ed8d9aa98 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:00.685456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde17bf3-dca8-4459-ac3b-7d2ed8d9aa98 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:00.689261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3a3c5-a29e-4399-93f1-c01700d0f9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:00.690113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3a3c5-a29e-4399-93f1-c01700d0f9ad None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:00.690460 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3a3c5-a29e-4399-93f1-c01700d0f9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:00.690809 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3a3c5-a29e-4399-93f1-c01700d0f9ad None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:00.691560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3a3c5-a29e-4399-93f1-c01700d0f9ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:00.693166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3a3c5-a29e-4399-93f1-c01700d0f9ad None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:00.693727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 496/994] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 22:21:00 2022] GET /identity/v3/endpoints?service_id=5e59d451e9ff4295856d2ab0fb976d06&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:00.701415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde17bf3-dca8-4459-ac3b-7d2ed8d9aa98 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:00.706085 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 499/995] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 22:21:00 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:21:00.817606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:00.825522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde17bf3-dca8-4459-ac3b-7d2ed8d9aa98 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:00.864862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cc50f-8460-4b8e-b5aa-fa814e6b274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:00.865822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cc50f-8460-4b8e-b5aa-fa814e6b274a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:00.866277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cc50f-8460-4b8e-b5aa-fa814e6b274a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:00.866713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cc50f-8460-4b8e-b5aa-fa814e6b274a None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:00.868817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cc50f-8460-4b8e-b5aa-fa814e6b274a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:00.870063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cc50f-8460-4b8e-b5aa-fa814e6b274a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:00.878201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 497/996] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 22:21:00 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:21:01.302526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e5b5a-90f9-4860-a406-477865b63811 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:01.303443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e5b5a-90f9-4860-a406-477865b63811 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:01.303443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e5b5a-90f9-4860-a406-477865b63811 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:01.304538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 500/997] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:01.311076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d93833-b6cf-4bde-a02d-150d4b765aad None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:01.311369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d93833-b6cf-4bde-a02d-150d4b765aad None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:01.311647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d93833-b6cf-4bde-a02d-150d4b765aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:01.330224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-b7d93833-b6cf-4bde-a02d-150d4b765aad None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:01.331420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b7d93833-b6cf-4bde-a02d-150d4b765aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:01.408426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 498/998] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:21:01.437294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e5b5a-90f9-4860-a406-477865b63811 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:01.609299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e5b5a-90f9-4860-a406-477865b63811 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:01.657307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a984fb-1e2e-48de-9754-fdfa5a008f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:01.658115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a984fb-1e2e-48de-9754-fdfa5a008f8f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:01.658412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a984fb-1e2e-48de-9754-fdfa5a008f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:01.658662 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a984fb-1e2e-48de-9754-fdfa5a008f8f None admin] PATH_INFO: `/v3/services/image` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:01.659325 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a984fb-1e2e-48de-9754-fdfa5a008f8f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:01.660758 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a984fb-1e2e-48de-9754-fdfa5a008f8f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-33a984fb-1e2e-48de-9754-fdfa5a008f8f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:01.673129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:01.674208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:01.675100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 501/999] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Jul 21 22:21:01 2022] GET /identity/v3/services/image => generated 86 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:01.681007 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d93833-b6cf-4bde-a02d-150d4b765aad None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:01.779765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:01.791328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d93833-b6cf-4bde-a02d-150d4b765aad None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:01.827125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa88df-1d03-4494-9a45-383c5b06b15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:01.827889 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa88df-1d03-4494-9a45-383c5b06b15f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:01.828079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa88df-1d03-4494-9a45-383c5b06b15f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:01.828355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa88df-1d03-4494-9a45-383c5b06b15f None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:01.829106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa88df-1d03-4494-9a45-383c5b06b15f None admin] RBAC: Adding query filter params (name=image) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:01.829419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa88df-1d03-4494-9a45-383c5b06b15f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:01.830775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa88df-1d03-4494-9a45-383c5b06b15f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:01.841058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 499/1000] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:01.842149 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a984fb-1e2e-48de-9754-fdfa5a008f8f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:01.855087 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:01.954204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a984fb-1e2e-48de-9754-fdfa5a008f8f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:01.990355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d9fcd-9c20-4386-b1d3-027843bc5f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:01.991036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d9fcd-9c20-4386-b1d3-027843bc5f41 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:01.991318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d9fcd-9c20-4386-b1d3-027843bc5f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:01.991594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d9fcd-9c20-4386-b1d3-027843bc5f41 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:01.992300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d9fcd-9c20-4386-b1d3-027843bc5f41 None admin] RBAC: Adding query filter params (type=image) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:01.992639 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d9fcd-9c20-4386-b1d3-027843bc5f41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:01.994739 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d9fcd-9c20-4386-b1d3-027843bc5f41 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:02.008516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 502/1001] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:02.844978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057a1e0-688a-4151-a4c4-cb3ba34fd8c1 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:02.845498 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057a1e0-688a-4151-a4c4-cb3ba34fd8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:02.845577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057a1e0-688a-4151-a4c4-cb3ba34fd8c1 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:02.847067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 500/1002] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:02.852748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de03ae-5969-47d3-9386-4e5c4b446e50 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:02.853636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de03ae-5969-47d3-9386-4e5c4b446e50 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:02.853926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de03ae-5969-47d3-9386-4e5c4b446e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:02.871643 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d9de03ae-5969-47d3-9386-4e5c4b446e50 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:02.873191 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d9de03ae-5969-47d3-9386-4e5c4b446e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:02.941980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 503/1003] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:02.951177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7630d2b-d908-454c-aac7-5932023706eb None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:02.951462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7630d2b-d908-454c-aac7-5932023706eb None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:02.951745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7630d2b-d908-454c-aac7-5932023706eb None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:02.953125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 501/1004] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:02.958190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7dcea-67be-4133-976e-f08fe8a3357a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:02.958494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7dcea-67be-4133-976e-f08fe8a3357a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:02.958770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7dcea-67be-4133-976e-f08fe8a3357a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:02.964734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:02.967663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7630d2b-d908-454c-aac7-5932023706eb None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:02.978193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d0b7dcea-67be-4133-976e-f08fe8a3357a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:02.979389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d0b7dcea-67be-4133-976e-f08fe8a3357a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:03.053896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 504/1005] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:03.078561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7dcea-67be-4133-976e-f08fe8a3357a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:03.084151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7630d2b-d908-454c-aac7-5932023706eb None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:03.123440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78894926-c5e6-4c99-a0ae-c80a89dc5bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:03.124215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78894926-c5e6-4c99-a0ae-c80a89dc5bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:03.124520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78894926-c5e6-4c99-a0ae-c80a89dc5bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:03.124812 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78894926-c5e6-4c99-a0ae-c80a89dc5bd3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:03.125963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78894926-c5e6-4c99-a0ae-c80a89dc5bd3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:03.127783 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78894926-c5e6-4c99-a0ae-c80a89dc5bd3 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-78894926-c5e6-4c99-a0ae-c80a89dc5bd3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:03.141202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:03.147884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:03.148877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 502/1006] 10.222.0.42 () {56 vars in 1126 bytes} [Thu Jul 21 22:21:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:03.150789 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78894926-c5e6-4c99-a0ae-c80a89dc5bd3 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:03.171872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:03.193839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7dcea-67be-4133-976e-f08fe8a3357a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:03.228052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:03.234465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084b380-6726-4f90-8ed6-aa76a75c5fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:03.235274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084b380-6726-4f90-8ed6-aa76a75c5fdf None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:03.235560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084b380-6726-4f90-8ed6-aa76a75c5fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:03.235837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084b380-6726-4f90-8ed6-aa76a75c5fdf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:03.236630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084b380-6726-4f90-8ed6-aa76a75c5fdf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:03.237915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084b380-6726-4f90-8ed6-aa76a75c5fdf None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-1084b380-6726-4f90-8ed6-aa76a75c5fdf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:03.251109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:03.252366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:03.253274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 505/1007] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:21:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:03.258362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084b380-6726-4f90-8ed6-aa76a75c5fdf None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:03.269270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78894926-c5e6-4c99-a0ae-c80a89dc5bd3 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:03.306474 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ea80f-afd2-4193-b99f-369bfb5ad31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:03.307160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ea80f-afd2-4193-b99f-369bfb5ad31d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:03.307453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ea80f-afd2-4193-b99f-369bfb5ad31d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:03.307729 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ea80f-afd2-4193-b99f-369bfb5ad31d None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:03.308442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ea80f-afd2-4193-b99f-369bfb5ad31d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:03.308795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ea80f-afd2-4193-b99f-369bfb5ad31d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:03.310159 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ea80f-afd2-4193-b99f-369bfb5ad31d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:03.315020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 503/1008] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 22:21:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:03.320621 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ea80f-afd2-4193-b99f-369bfb5ad31d None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:03.374028 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084b380-6726-4f90-8ed6-aa76a75c5fdf None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:03.410162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:03.410525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255abbd-f2f8-4b7c-9164-2b562d2b60d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:03.411161 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255abbd-f2f8-4b7c-9164-2b562d2b60d0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:03.411449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255abbd-f2f8-4b7c-9164-2b562d2b60d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:03.411724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255abbd-f2f8-4b7c-9164-2b562d2b60d0 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:03.412645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255abbd-f2f8-4b7c-9164-2b562d2b60d0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:03.412793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255abbd-f2f8-4b7c-9164-2b562d2b60d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:03.416974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255abbd-f2f8-4b7c-9164-2b562d2b60d0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:03.426123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 506/1009] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:21:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:03.427241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255abbd-f2f8-4b7c-9164-2b562d2b60d0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:03.443331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ea80f-afd2-4193-b99f-369bfb5ad31d None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:03.479773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53484a-e594-4606-a812-5a7db7bca434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:03.480456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53484a-e594-4606-a812-5a7db7bca434 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:03.480772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53484a-e594-4606-a812-5a7db7bca434 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:03.481055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53484a-e594-4606-a812-5a7db7bca434 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:03.481773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53484a-e594-4606-a812-5a7db7bca434 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:03.482081 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53484a-e594-4606-a812-5a7db7bca434 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:03.483461 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53484a-e594-4606-a812-5a7db7bca434 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:03.488758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 504/1010] 10.222.0.42 () {56 vars in 1122 bytes} [Thu Jul 21 22:21:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:03.494251 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:03.495022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53484a-e594-4606-a812-5a7db7bca434 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:03.541891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255abbd-f2f8-4b7c-9164-2b562d2b60d0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:03.576770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd5335-254f-4414-9875-3ce7012f5903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:03.577445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd5335-254f-4414-9875-3ce7012f5903 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:03.577732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd5335-254f-4414-9875-3ce7012f5903 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:03.578007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd5335-254f-4414-9875-3ce7012f5903 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:03.578723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd5335-254f-4414-9875-3ce7012f5903 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:03.579026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd5335-254f-4414-9875-3ce7012f5903 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:03.580431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd5335-254f-4414-9875-3ce7012f5903 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:03.591120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 507/1011] 10.222.0.42 () {56 vars in 1112 bytes} [Thu Jul 21 22:21:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:03.592337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd5335-254f-4414-9875-3ce7012f5903 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:03.612006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53484a-e594-4606-a812-5a7db7bca434 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:03.655795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bb897-aa32-4f8e-9abd-5e71bd1a5581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:03.657024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bb897-aa32-4f8e-9abd-5e71bd1a5581 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:03.657701 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bb897-aa32-4f8e-9abd-5e71bd1a5581 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:03.658408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bb897-aa32-4f8e-9abd-5e71bd1a5581 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:03.659649 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bb897-aa32-4f8e-9abd-5e71bd1a5581 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:03.662388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bb897-aa32-4f8e-9abd-5e71bd1a5581 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:03.692911 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 505/1012] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 22:21:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:21:03.724135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd5335-254f-4414-9875-3ce7012f5903 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:03.752033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:03.773304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719e75c-ec0e-453d-acc4-10b0ea22acb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:03.774042 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e75c-ec0e-453d-acc4-10b0ea22acb2 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:03.774952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e75c-ec0e-453d-acc4-10b0ea22acb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:03.775590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e75c-ec0e-453d-acc4-10b0ea22acb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:03.775889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719e75c-ec0e-453d-acc4-10b0ea22acb2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:03.777325 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719e75c-ec0e-453d-acc4-10b0ea22acb2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:03.801991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 508/1013] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 22:21:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:21:04.096142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c020104-3d33-479a-bf9a-6ffd77e7bcb9 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:04.096695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c020104-3d33-479a-bf9a-6ffd77e7bcb9 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:04.096775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c020104-3d33-479a-bf9a-6ffd77e7bcb9 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:04.098171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 506/1014] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:04.105033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73208dfa-f590-4117-9ac6-f5811aefbc7f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:04.105591 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73208dfa-f590-4117-9ac6-f5811aefbc7f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:04.105591 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73208dfa-f590-4117-9ac6-f5811aefbc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:04.125240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-73208dfa-f590-4117-9ac6-f5811aefbc7f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:04.127168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-73208dfa-f590-4117-9ac6-f5811aefbc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:04.217445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 509/1015] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:04 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:21:04.237072 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c020104-3d33-479a-bf9a-6ffd77e7bcb9 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:04.247963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:04.362873 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c020104-3d33-479a-bf9a-6ffd77e7bcb9 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:04.397674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fc56c-4180-4ad0-b317-34063d8284d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:04.398631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fc56c-4180-4ad0-b317-34063d8284d8 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:04.399088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fc56c-4180-4ad0-b317-34063d8284d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:04.399507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fc56c-4180-4ad0-b317-34063d8284d8 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:04.400378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fc56c-4180-4ad0-b317-34063d8284d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:04.401935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fc56c-4180-4ad0-b317-34063d8284d8 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:04.423019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 507/1016] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:21:04 2022] POST /identity/v3/services => generated 242 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:21:05.825004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267c59c-d389-4e32-94c0-514931647f3e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:05.825521 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267c59c-d389-4e32-94c0-514931647f3e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:05.825631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267c59c-d389-4e32-94c0-514931647f3e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:05.827073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 510/1017] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:05.832280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ef9b6-28f6-40b0-92d4-eeca8590aaf4 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:05.832732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ef9b6-28f6-40b0-92d4-eeca8590aaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:05.832898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ef9b6-28f6-40b0-92d4-eeca8590aaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:05.849694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-0f3ef9b6-28f6-40b0-92d4-eeca8590aaf4 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:05.850871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0f3ef9b6-28f6-40b0-92d4-eeca8590aaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:05.914936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:05.931456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 508/1018] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:05.955116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267c59c-d389-4e32-94c0-514931647f3e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:06.015691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc8e08-d81f-431b-9a01-3e553b4010eb None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.016041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc8e08-d81f-431b-9a01-3e553b4010eb None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.016265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc8e08-d81f-431b-9a01-3e553b4010eb None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.017797 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 509/1019] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:21:06.022719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d84e8a-eda2-425b-9a92-771e7299c512 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.023011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d84e8a-eda2-425b-9a92-771e7299c512 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.023298 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d84e8a-eda2-425b-9a92-771e7299c512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.040843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-a4d84e8a-eda2-425b-9a92-771e7299c512 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:06.042046 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a4d84e8a-eda2-425b-9a92-771e7299c512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:06.063544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267c59c-d389-4e32-94c0-514931647f3e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:06.065168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:06.102094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45357a8d-0104-4927-be11-8c70d8d76770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:06.102872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45357a8d-0104-4927-be11-8c70d8d76770 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.103154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45357a8d-0104-4927-be11-8c70d8d76770 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.103424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45357a8d-0104-4927-be11-8c70d8d76770 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.108522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45357a8d-0104-4927-be11-8c70d8d76770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:06.108522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45357a8d-0104-4927-be11-8c70d8d76770 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:06.108522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 511/1020] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:06.114174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45357a8d-0104-4927-be11-8c70d8d76770 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:06.120677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 510/1021] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:21:06.144712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d84e8a-eda2-425b-9a92-771e7299c512 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:06.225714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45357a8d-0104-4927-be11-8c70d8d76770 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:06.255507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d84e8a-eda2-425b-9a92-771e7299c512 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:06.262214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00c7a7-44a2-490b-86e7-dfbea4049f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:06.262889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00c7a7-44a2-490b-86e7-dfbea4049f35 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.263171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00c7a7-44a2-490b-86e7-dfbea4049f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.263443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00c7a7-44a2-490b-86e7-dfbea4049f35 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.264156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00c7a7-44a2-490b-86e7-dfbea4049f35 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:06.265635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00c7a7-44a2-490b-86e7-dfbea4049f35 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:06.281709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:06.294710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 512/1022] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:21:06 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:21:06.306368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483a2cb-27b2-42e0-85b9-00a85d6a81cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:06.307423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483a2cb-27b2-42e0-85b9-00a85d6a81cf None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.307714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483a2cb-27b2-42e0-85b9-00a85d6a81cf None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.307997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483a2cb-27b2-42e0-85b9-00a85d6a81cf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.322712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483a2cb-27b2-42e0-85b9-00a85d6a81cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-a483a2cb-27b2-42e0-85b9-00a85d6a81cf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:06.331343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:06.332447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:06.333484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:06.333484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:06.333484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:06.333484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:06.333484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:06.333484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:06.333484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:06.333484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:06.333484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:06.333484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:06.333484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:21:06.333484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:06.337023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 511/1023] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jul 21 22:21:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:06.339642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00c7a7-44a2-490b-86e7-dfbea4049f35 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:06.372825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:06.478778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00c7a7-44a2-490b-86e7-dfbea4049f35 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:06.522218 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08de0d-5bf7-48e5-bcd6-2d4e0c6204a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:06.523163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08de0d-5bf7-48e5-bcd6-2d4e0c6204a0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.523450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08de0d-5bf7-48e5-bcd6-2d4e0c6204a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.523764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08de0d-5bf7-48e5-bcd6-2d4e0c6204a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.524037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc075e8-1ca6-46a4-9c1d-c4c9d8057b0a None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.524337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc075e8-1ca6-46a4-9c1d-c4c9d8057b0a None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.524574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08de0d-5bf7-48e5-bcd6-2d4e0c6204a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:06.524681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc075e8-1ca6-46a4-9c1d-c4c9d8057b0a None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.524891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08de0d-5bf7-48e5-bcd6-2d4e0c6204a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:06.526292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08de0d-5bf7-48e5-bcd6-2d4e0c6204a0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:06.526524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 512/1024] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:21:06.532527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6ef31-3fc9-4720-aa1f-8b6a1aa6e1dd None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.532748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6ef31-3fc9-4720-aa1f-8b6a1aa6e1dd None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.533001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6ef31-3fc9-4720-aa1f-8b6a1aa6e1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.536370 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 513/1025] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 22:21:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:06.543383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08de0d-5bf7-48e5-bcd6-2d4e0c6204a0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:06.552283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-0ef6ef31-3fc9-4720-aa1f-8b6a1aa6e1dd None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:06.553563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0ef6ef31-3fc9-4720-aa1f-8b6a1aa6e1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:06.636749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 513/1026] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:06.653007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:06.660221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6ef31-3fc9-4720-aa1f-8b6a1aa6e1dd None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:06.677818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08de0d-5bf7-48e5-bcd6-2d4e0c6204a0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:06.719975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c14538-78d2-42c3-8d06-70896940620d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:06.720813 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c14538-78d2-42c3-8d06-70896940620d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.721511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c14538-78d2-42c3-8d06-70896940620d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.721511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c14538-78d2-42c3-8d06-70896940620d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.733980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c14538-78d2-42c3-8d06-70896940620d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:06.735456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c14538-78d2-42c3-8d06-70896940620d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-32c14538-78d2-42c3-8d06-70896940620d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:21:06.755800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:06.757202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:06.759776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:21:06.759776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:21:06.759776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:06.759776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:06.759776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 514/1027] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:06.763898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c14538-78d2-42c3-8d06-70896940620d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:06.786407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6ef31-3fc9-4720-aa1f-8b6a1aa6e1dd None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:06.801093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:06.828374 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fe2a8-6dbd-40ae-833b-0a08b5131fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:06.829226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fe2a8-6dbd-40ae-833b-0a08b5131fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.829515 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fe2a8-6dbd-40ae-833b-0a08b5131fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.829797 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fe2a8-6dbd-40ae-833b-0a08b5131fb8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.830468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fe2a8-6dbd-40ae-833b-0a08b5131fb8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:06.831854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fe2a8-6dbd-40ae-833b-0a08b5131fb8 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-f77fe2a8-6dbd-40ae-833b-0a08b5131fb8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:06.847233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:06.850412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:06.852545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 514/1028] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Jul 21 22:21:06 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:06.855733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fe2a8-6dbd-40ae-833b-0a08b5131fb8 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:06.881360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c14538-78d2-42c3-8d06-70896940620d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:06.923609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fe103-5b92-4050-bac0-0123a98fb7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:06.924545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fe103-5b92-4050-bac0-0123a98fb7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:06.924836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fe103-5b92-4050-bac0-0123a98fb7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:06.925111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fe103-5b92-4050-bac0-0123a98fb7a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:06.925855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fe103-5b92-4050-bac0-0123a98fb7a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:06.926151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fe103-5b92-4050-bac0-0123a98fb7a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:06.927540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fe103-5b92-4050-bac0-0123a98fb7a0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:06.932357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:06.944365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 515/1029] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:06.951443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fe103-5b92-4050-bac0-0123a98fb7a0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:06.982388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fe2a8-6dbd-40ae-833b-0a08b5131fb8 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:07.022022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822a2cd-9706-4a4d-92d0-2c4cb257bb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:07.022704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822a2cd-9706-4a4d-92d0-2c4cb257bb19 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:07.023010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822a2cd-9706-4a4d-92d0-2c4cb257bb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:07.023276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822a2cd-9706-4a4d-92d0-2c4cb257bb19 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:07.024268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822a2cd-9706-4a4d-92d0-2c4cb257bb19 None admin] RBAC: Adding query filter params (name=image) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:07.024804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822a2cd-9706-4a4d-92d0-2c4cb257bb19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:07.026005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822a2cd-9706-4a4d-92d0-2c4cb257bb19 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:07.031510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 515/1030] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:07.037181 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822a2cd-9706-4a4d-92d0-2c4cb257bb19 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:07.081698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fe103-5b92-4050-bac0-0123a98fb7a0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:07.091327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:07.125838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790c48e-1bbf-451a-abec-0a0bafbb5106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:07.126758 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790c48e-1bbf-451a-abec-0a0bafbb5106 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:07.127045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790c48e-1bbf-451a-abec-0a0bafbb5106 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:07.127317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790c48e-1bbf-451a-abec-0a0bafbb5106 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:07.135706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790c48e-1bbf-451a-abec-0a0bafbb5106 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:07.137244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790c48e-1bbf-451a-abec-0a0bafbb5106 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-4790c48e-1bbf-451a-abec-0a0bafbb5106 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:21:07.152511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:07.153931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:07.154920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:07.154920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:07.154920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:21:07.154920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:21:07.154920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:21:07.154920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:21:07.154920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:21:07.154920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:21:07.154920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:21:07.154920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:07.154920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 516/1031] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 22:21:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:07.159726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790c48e-1bbf-451a-abec-0a0bafbb5106 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:07.169017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822a2cd-9706-4a4d-92d0-2c4cb257bb19 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:07.208576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591f8de-7b5c-41f0-ad3c-337842c9fbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:07.209266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591f8de-7b5c-41f0-ad3c-337842c9fbeb None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:07.209595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591f8de-7b5c-41f0-ad3c-337842c9fbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:07.209987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591f8de-7b5c-41f0-ad3c-337842c9fbeb None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:07.210556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591f8de-7b5c-41f0-ad3c-337842c9fbeb None admin] RBAC: Adding query filter params (type=image) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:07.210867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591f8de-7b5c-41f0-ad3c-337842c9fbeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:07.212430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591f8de-7b5c-41f0-ad3c-337842c9fbeb None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:07.217934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 516/1032] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:07.223113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:07.224226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591f8de-7b5c-41f0-ad3c-337842c9fbeb None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:07.283076 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790c48e-1bbf-451a-abec-0a0bafbb5106 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:07.327362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6e112-b261-42cb-a4de-2a1b1ebdf6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:07.328026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6e112-b261-42cb-a4de-2a1b1ebdf6ab None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:07.328312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6e112-b261-42cb-a4de-2a1b1ebdf6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:07.328620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6e112-b261-42cb-a4de-2a1b1ebdf6ab None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:07.329468 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6e112-b261-42cb-a4de-2a1b1ebdf6ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:07.329769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6e112-b261-42cb-a4de-2a1b1ebdf6ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:07.331214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6e112-b261-42cb-a4de-2a1b1ebdf6ab None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:07.346496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591f8de-7b5c-41f0-ad3c-337842c9fbeb None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:07.349515 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:07.351338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 517/1033] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 22:21:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:07.358990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6e112-b261-42cb-a4de-2a1b1ebdf6ab None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:07.391642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a869f8-2f95-43a3-8e36-7af039d7d188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:07.392423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a869f8-2f95-43a3-8e36-7af039d7d188 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:07.392736 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a869f8-2f95-43a3-8e36-7af039d7d188 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:07.393011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a869f8-2f95-43a3-8e36-7af039d7d188 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:07.394136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a869f8-2f95-43a3-8e36-7af039d7d188 None admin] RBAC: Adding query filter params (interface=public, service_id=c31852f1f3ed429cad5534d50ecbcaa9, region_id=RegionOne) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:07.394471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a869f8-2f95-43a3-8e36-7af039d7d188 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:07.395840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a869f8-2f95-43a3-8e36-7af039d7d188 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:07.406647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 517/1034] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 22:21:07 2022] GET /identity/v3/endpoints?service_id=c31852f1f3ed429cad5534d50ecbcaa9&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:07.407980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a869f8-2f95-43a3-8e36-7af039d7d188 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:07.479991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6e112-b261-42cb-a4de-2a1b1ebdf6ab None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:07.499350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:07.524748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e93057-1d6c-411f-a481-ae716324a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:07.525529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e93057-1d6c-411f-a481-ae716324a205 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:07.526065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e93057-1d6c-411f-a481-ae716324a205 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:07.526065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e93057-1d6c-411f-a481-ae716324a205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:07.529880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a869f8-2f95-43a3-8e36-7af039d7d188 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:07.572709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f1b84-366f-421c-bbd0-ce595d26d063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:07.573878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f1b84-366f-421c-bbd0-ce595d26d063 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:07.574411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f1b84-366f-421c-bbd0-ce595d26d063 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:07.574932 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f1b84-366f-421c-bbd0-ce595d26d063 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:07.576275 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f1b84-366f-421c-bbd0-ce595d26d063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:07.578973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f1b84-366f-421c-bbd0-ce595d26d063 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:07.592552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 518/1035] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 22:21:07 2022] GET /identity/v3/services => generated 1957 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:21:07.619033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e93057-1d6c-411f-a481-ae716324a205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:07.620583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e93057-1d6c-411f-a481-ae716324a205 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:07.751917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 518/1036] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:21:07.757920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f1b84-366f-421c-bbd0-ce595d26d063 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:07.794132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:07.873265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f1b84-366f-421c-bbd0-ce595d26d063 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:07.908624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6833a37-cab8-469a-a21a-0a82902d6d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:07.909513 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6833a37-cab8-469a-a21a-0a82902d6d6e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:07.909806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6833a37-cab8-469a-a21a-0a82902d6d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:07.910086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6833a37-cab8-469a-a21a-0a82902d6d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:07.920563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6833a37-cab8-469a-a21a-0a82902d6d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:07.922015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6833a37-cab8-469a-a21a-0a82902d6d6e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-f6833a37-cab8-469a-a21a-0a82902d6d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:21:07.940395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:07.941394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:07.947657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:21:07.947657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:21:07.947657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:07.947657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:07.947657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 519/1037] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:07.949578 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e93057-1d6c-411f-a481-ae716324a205 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:07.966069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:08.056549 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e93057-1d6c-411f-a481-ae716324a205 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:08.091780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87be59-d70e-4971-a0b9-7414ccac883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:08.092455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87be59-d70e-4971-a0b9-7414ccac883f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:08.092772 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87be59-d70e-4971-a0b9-7414ccac883f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:08.093053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87be59-d70e-4971-a0b9-7414ccac883f None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:08.093786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87be59-d70e-4971-a0b9-7414ccac883f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:08.094096 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87be59-d70e-4971-a0b9-7414ccac883f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:08.095473 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87be59-d70e-4971-a0b9-7414ccac883f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:08.108178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 519/1038] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:08.114579 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6833a37-cab8-469a-a21a-0a82902d6d6e None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:08.202768 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:08.221171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6833a37-cab8-469a-a21a-0a82902d6d6e None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:08.256827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd8144-bae9-4c39-a722-4b1f45cc6b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:08.257631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd8144-bae9-4c39-a722-4b1f45cc6b6c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:08.257926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd8144-bae9-4c39-a722-4b1f45cc6b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:08.258305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd8144-bae9-4c39-a722-4b1f45cc6b6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:08.265399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd8144-bae9-4c39-a722-4b1f45cc6b6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:08.266837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd8144-bae9-4c39-a722-4b1f45cc6b6c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-86cd8144-bae9-4c39-a722-4b1f45cc6b6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:21:08.281229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:08.282173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:08.283096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:21:08.283096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:21:08.283096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:21:08.283096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:21:08.283096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:21:08.283096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:21:08.283096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:21:08.283096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:08.287349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 520/1039] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:21:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:08.288796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87be59-d70e-4971-a0b9-7414ccac883f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:08.371271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:08.398190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87be59-d70e-4971-a0b9-7414ccac883f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:08.439142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29236b98-8b4c-4777-947c-c357cdb2d300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:08.439815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29236b98-8b4c-4777-947c-c357cdb2d300 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:08.440489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29236b98-8b4c-4777-947c-c357cdb2d300 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:08.440489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29236b98-8b4c-4777-947c-c357cdb2d300 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:08.441290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29236b98-8b4c-4777-947c-c357cdb2d300 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:08.441603 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29236b98-8b4c-4777-947c-c357cdb2d300 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:08.441989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cb7be-06e9-4f61-bbd8-bd41115e0cde None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:08.442283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cb7be-06e9-4f61-bbd8-bd41115e0cde None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:08.442558 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2cb7be-06e9-4f61-bbd8-bd41115e0cde None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:08.443049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29236b98-8b4c-4777-947c-c357cdb2d300 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:08.444024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 521/1040] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:08.448813 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53920bf8-7564-4974-b087-31d0ff00edef None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:08.449111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53920bf8-7564-4974-b087-31d0ff00edef None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:08.449390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53920bf8-7564-4974-b087-31d0ff00edef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:08.456437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 520/1041] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:21:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:08.462456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29236b98-8b4c-4777-947c-c357cdb2d300 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:08.467741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-53920bf8-7564-4974-b087-31d0ff00edef None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:08.468973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-53920bf8-7564-4974-b087-31d0ff00edef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:08.543953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 522/1042] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:21:08.567210 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29236b98-8b4c-4777-947c-c357cdb2d300 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:08.568827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53920bf8-7564-4974-b087-31d0ff00edef None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:08.605071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482cf1a-69ed-49a5-a15c-809e1d811a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:08.605885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482cf1a-69ed-49a5-a15c-809e1d811a21 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:08.606173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482cf1a-69ed-49a5-a15c-809e1d811a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:08.606485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482cf1a-69ed-49a5-a15c-809e1d811a21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:08.607941 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482cf1a-69ed-49a5-a15c-809e1d811a21 None admin] RBAC: Adding query filter params (role.id=803aed498ea04514b286c6e64ee985cd, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=c77f6ccc7cb84c04ac532007a04a0f5e) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:08.608332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482cf1a-69ed-49a5-a15c-809e1d811a21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:08.610094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482cf1a-69ed-49a5-a15c-809e1d811a21 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:08.612182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:08.618957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 521/1043] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:08 2022] GET /identity/v3/role_assignments?user.id=c77f6ccc7cb84c04ac532007a04a0f5e&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=803aed498ea04514b286c6e64ee985cd => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:08.656121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:08.686876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53920bf8-7564-4974-b087-31d0ff00edef None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:08.726314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9328e0e-d4e2-4c82-8ef2-dc8f92f0c253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:08.727577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9328e0e-d4e2-4c82-8ef2-dc8f92f0c253 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:08.728022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9328e0e-d4e2-4c82-8ef2-dc8f92f0c253 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:08.728554 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9328e0e-d4e2-4c82-8ef2-dc8f92f0c253 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:08.730011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9328e0e-d4e2-4c82-8ef2-dc8f92f0c253 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:08.731647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9328e0e-d4e2-4c82-8ef2-dc8f92f0c253 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:08.738120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 523/1044] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:08.739906 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482cf1a-69ed-49a5-a15c-809e1d811a21 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:08.859747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482cf1a-69ed-49a5-a15c-809e1d811a21 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:08.894746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8017e3-c4fa-47a9-b935-da0d113ec06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:08.895423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8017e3-c4fa-47a9-b935-da0d113ec06e None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:08.895711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8017e3-c4fa-47a9-b935-da0d113ec06e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:08.895992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8017e3-c4fa-47a9-b935-da0d113ec06e None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:08.896749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8017e3-c4fa-47a9-b935-da0d113ec06e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:08.898188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8017e3-c4fa-47a9-b935-da0d113ec06e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:08.927580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 522/1045] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:21:08 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:21:09.761805 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf6aea-7b61-4265-9ba6-7ccfb96fb425 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:09.762350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf6aea-7b61-4265-9ba6-7ccfb96fb425 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:09.762350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf6aea-7b61-4265-9ba6-7ccfb96fb425 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:09.763779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 524/1046] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:21:09.769501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626e2cc-d6ed-4a82-8169-15f9c6ea25b6 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:09.769959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626e2cc-d6ed-4a82-8169-15f9c6ea25b6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:09.770164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626e2cc-d6ed-4a82-8169-15f9c6ea25b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:09.787907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7626e2cc-d6ed-4a82-8169-15f9c6ea25b6 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:09.789656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7626e2cc-d6ed-4a82-8169-15f9c6ea25b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:09.846507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:09.868801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 523/1047] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:09.890326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf6aea-7b61-4265-9ba6-7ccfb96fb425 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:10.004866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf6aea-7b61-4265-9ba6-7ccfb96fb425 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:10.041538 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb7996-0d13-47bc-aa5d-d59c5907661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:10.042343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb7996-0d13-47bc-aa5d-d59c5907661c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:10.042714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb7996-0d13-47bc-aa5d-d59c5907661c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:10.043003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb7996-0d13-47bc-aa5d-d59c5907661c None admin] PATH_INFO: `/v3/services/image` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:10.043667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb7996-0d13-47bc-aa5d-d59c5907661c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:10.045075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb7996-0d13-47bc-aa5d-d59c5907661c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-b7bb7996-0d13-47bc-aa5d-d59c5907661c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:10.057032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:10.058177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:10.059099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 525/1048] 10.222.0.42 () {56 vars in 1105 bytes} [Thu Jul 21 22:21:09 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:10.066813 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626e2cc-d6ed-4a82-8169-15f9c6ea25b6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:10.181656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626e2cc-d6ed-4a82-8169-15f9c6ea25b6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:10.216684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da93751-28e0-40d5-b7d2-780fc7929603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:10.217555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da93751-28e0-40d5-b7d2-780fc7929603 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:10.217713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da93751-28e0-40d5-b7d2-780fc7929603 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:10.217966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da93751-28e0-40d5-b7d2-780fc7929603 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:10.218636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da93751-28e0-40d5-b7d2-780fc7929603 None admin] RBAC: Adding query filter params (name=image) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:10.218921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da93751-28e0-40d5-b7d2-780fc7929603 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:10.220313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da93751-28e0-40d5-b7d2-780fc7929603 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:10.231436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 524/1049] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:10.232765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb7996-0d13-47bc-aa5d-d59c5907661c None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:10.243355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:10.352217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb7996-0d13-47bc-aa5d-d59c5907661c None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:10.389212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa4917-a9d9-4177-8d2a-4555a1aac0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:10.389890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa4917-a9d9-4177-8d2a-4555a1aac0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:10.390177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa4917-a9d9-4177-8d2a-4555a1aac0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:10.390455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa4917-a9d9-4177-8d2a-4555a1aac0d7 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:10.391171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa4917-a9d9-4177-8d2a-4555a1aac0d7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:10.391478 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa4917-a9d9-4177-8d2a-4555a1aac0d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:10.392882 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa4917-a9d9-4177-8d2a-4555a1aac0d7 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:10.403211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 526/1050] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:10.404405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da93751-28e0-40d5-b7d2-780fc7929603 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:10.461119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:10.515297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da93751-28e0-40d5-b7d2-780fc7929603 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:10.550949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5eddf-12c9-418c-88c6-b348dfbd6894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:10.551613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5eddf-12c9-418c-88c6-b348dfbd6894 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:10.551893 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5eddf-12c9-418c-88c6-b348dfbd6894 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:10.552168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5eddf-12c9-418c-88c6-b348dfbd6894 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:10.552974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5eddf-12c9-418c-88c6-b348dfbd6894 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:10.554323 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5eddf-12c9-418c-88c6-b348dfbd6894 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:10.575973 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 525/1051] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 22:21:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:21:10.778310 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef76984-2647-4f43-8714-f043c02ed22e None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:10.778851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef76984-2647-4f43-8714-f043c02ed22e None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:10.778851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef76984-2647-4f43-8714-f043c02ed22e None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:10.780150 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 527/1052] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:10.786227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69132ae4-9f0c-4d9f-aa5b-af253065a5e7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:10.786721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69132ae4-9f0c-4d9f-aa5b-af253065a5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:10.786832 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69132ae4-9f0c-4d9f-aa5b-af253065a5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:10.806360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-69132ae4-9f0c-4d9f-aa5b-af253065a5e7 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:10.807583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-69132ae4-9f0c-4d9f-aa5b-af253065a5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:10.896528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 526/1053] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:10.919295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef76984-2647-4f43-8714-f043c02ed22e None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:11.007876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:11.043830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef76984-2647-4f43-8714-f043c02ed22e None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:11.079402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21dcda-8f7c-4599-ab12-76ad0fd0d7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:11.080225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21dcda-8f7c-4599-ab12-76ad0fd0d7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:11.080536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21dcda-8f7c-4599-ab12-76ad0fd0d7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:11.080816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21dcda-8f7c-4599-ab12-76ad0fd0d7e0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:11.092006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21dcda-8f7c-4599-ab12-76ad0fd0d7e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:11.098281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35dbfb-4b82-4a66-abdf-3195e9f671b6 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:11.098580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35dbfb-4b82-4a66-abdf-3195e9f671b6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:11.098864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35dbfb-4b82-4a66-abdf-3195e9f671b6 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:11.100247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 527/1054] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-0d21dcda-8f7c-4599-ab12-76ad0fd0d7e0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:11.100388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:11.105323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:11.106289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 528/1055] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jul 21 22:21:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:11.107034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70570bb-b92d-4fd6-ba60-4b4746c675c8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:11.107034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70570bb-b92d-4fd6-ba60-4b4746c675c8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:11.107034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70570bb-b92d-4fd6-ba60-4b4746c675c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:11.108858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21dcda-8f7c-4599-ab12-76ad0fd0d7e0 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:11.124402 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b70570bb-b92d-4fd6-ba60-4b4746c675c8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:11.125650 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b70570bb-b92d-4fd6-ba60-4b4746c675c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:11.154797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:11.216797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 528/1056] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:21:11.219741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21dcda-8f7c-4599-ab12-76ad0fd0d7e0 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:11.228794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70570bb-b92d-4fd6-ba60-4b4746c675c8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:11.250652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:11.258390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d337bef-3157-4518-baba-f91d645f489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:11.259070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d337bef-3157-4518-baba-f91d645f489b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:11.259352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d337bef-3157-4518-baba-f91d645f489b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:11.259633 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d337bef-3157-4518-baba-f91d645f489b None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:11.260384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d337bef-3157-4518-baba-f91d645f489b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:11.260725 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d337bef-3157-4518-baba-f91d645f489b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:11.262098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d337bef-3157-4518-baba-f91d645f489b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:11.276670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 529/1057] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 22:21:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:11.277853 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d337bef-3157-4518-baba-f91d645f489b None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:11.346454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70570bb-b92d-4fd6-ba60-4b4746c675c8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:11.382530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d337bef-3157-4518-baba-f91d645f489b None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:11.384971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dff9b-8b6b-4769-81db-c913190d7e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:11.385876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dff9b-8b6b-4769-81db-c913190d7e7f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:11.386167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dff9b-8b6b-4769-81db-c913190d7e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:11.386449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dff9b-8b6b-4769-81db-c913190d7e7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:11.398714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6dff9b-8b6b-4769-81db-c913190d7e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-dd6dff9b-8b6b-4769-81db-c913190d7e7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:11.406523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:11.407710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:11.408627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 529/1058] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:11.413778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dff9b-8b6b-4769-81db-c913190d7e7f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:11.424837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39621d-a987-4a12-b9f7-966cf6aa85cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:11.426089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39621d-a987-4a12-b9f7-966cf6aa85cd None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:11.426389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39621d-a987-4a12-b9f7-966cf6aa85cd None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:11.426668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39621d-a987-4a12-b9f7-966cf6aa85cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:11.440751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39621d-a987-4a12-b9f7-966cf6aa85cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:11.441343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:11.442615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39621d-a987-4a12-b9f7-966cf6aa85cd None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-7b39621d-a987-4a12-b9f7-966cf6aa85cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:21:11.461585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:11.462563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:11.469222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:21:11.469222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:21:11.469222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:11.469222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:11.469222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 530/1059] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:11.469222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39621d-a987-4a12-b9f7-966cf6aa85cd None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:11.517474 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:11.538604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6dff9b-8b6b-4769-81db-c913190d7e7f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:11.576461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b757f5b-b721-4654-8960-7f4e48ddf3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:11.577152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b757f5b-b721-4654-8960-7f4e48ddf3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:11.577436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b757f5b-b721-4654-8960-7f4e48ddf3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:11.577706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b757f5b-b721-4654-8960-7f4e48ddf3e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:11.578451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b757f5b-b721-4654-8960-7f4e48ddf3e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:11.578754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b757f5b-b721-4654-8960-7f4e48ddf3e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:11.580119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b757f5b-b721-4654-8960-7f4e48ddf3e1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:11.581921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39621d-a987-4a12-b9f7-966cf6aa85cd None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:11.590026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 530/1060] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:11.595480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b757f5b-b721-4654-8960-7f4e48ddf3e1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:11.622823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be2c1f-c2e2-4376-a62b-99b30343c889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:11.623435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be2c1f-c2e2-4376-a62b-99b30343c889 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:11.623690 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be2c1f-c2e2-4376-a62b-99b30343c889 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:11.623964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be2c1f-c2e2-4376-a62b-99b30343c889 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:11.624719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be2c1f-c2e2-4376-a62b-99b30343c889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:11.625025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be2c1f-c2e2-4376-a62b-99b30343c889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:11.626375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be2c1f-c2e2-4376-a62b-99b30343c889 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:11.643903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 531/1061] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:11.644995 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be2c1f-c2e2-4376-a62b-99b30343c889 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:11.693646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:11.713825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b757f5b-b721-4654-8960-7f4e48ddf3e1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:11.752620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be2c1f-c2e2-4376-a62b-99b30343c889 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:11.753478 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134b838-039c-4108-8bef-11bc1991ea05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:11.754548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134b838-039c-4108-8bef-11bc1991ea05 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:11.754992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134b838-039c-4108-8bef-11bc1991ea05 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:11.755261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134b838-039c-4108-8bef-11bc1991ea05 None admin] PATH_INFO: `/v3/users/6bc11fa221a0403aa9a430034f4359c7` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:11.755875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:11.771198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134b838-039c-4108-8bef-11bc1991ea05 None admin] RBAC: Authorizing `identity:get_user(user_id=6bc11fa221a0403aa9a430034f4359c7)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:11.772767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134b838-039c-4108-8bef-11bc1991ea05 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:11.779100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 531/1062] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:11 2022] GET /identity/v3/users/6bc11fa221a0403aa9a430034f4359c7 => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:11.780215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134b838-039c-4108-8bef-11bc1991ea05 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:11.798135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ff145-3f91-407b-84b6-019787b53684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:11.798949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ff145-3f91-407b-84b6-019787b53684 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:11.799240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ff145-3f91-407b-84b6-019787b53684 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:11.799519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ff145-3f91-407b-84b6-019787b53684 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:11.808724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ff145-3f91-407b-84b6-019787b53684 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:11.810282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ff145-3f91-407b-84b6-019787b53684 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-281ff145-3f91-407b-84b6-019787b53684 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:21:11.826082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:11.827256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:11.828494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:11.828494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:11.828494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:21:11.828494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:21:11.828494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:21:11.828494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:21:11.828494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:21:11.828494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:21:11.828494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:21:11.828494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:11.831935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 532/1063] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 22:21:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:11.832868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ff145-3f91-407b-84b6-019787b53684 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:11.896374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134b838-039c-4108-8bef-11bc1991ea05 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:11.938448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c4f04-24af-4706-9a04-35dece4180b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:11.939207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c4f04-24af-4706-9a04-35dece4180b7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:11.939496 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c4f04-24af-4706-9a04-35dece4180b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:11.939776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c4f04-24af-4706-9a04-35dece4180b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:11.942711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ff145-3f91-407b-84b6-019787b53684 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:11.951776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:11.982640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a442d-e086-4b57-b3c4-4264604c9235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:11.983321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a442d-e086-4b57-b3c4-4264604c9235 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:11.983619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a442d-e086-4b57-b3c4-4264604c9235 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:11.983874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a442d-e086-4b57-b3c4-4264604c9235 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:11.985183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a442d-e086-4b57-b3c4-4264604c9235 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:11.985497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a442d-e086-4b57-b3c4-4264604c9235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:11.987291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a442d-e086-4b57-b3c4-4264604c9235 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:12.006367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 533/1064] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 22:21:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:12.012341 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a442d-e086-4b57-b3c4-4264604c9235 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:12.016794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c4f04-24af-4706-9a04-35dece4180b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:12.018611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c4f04-24af-4706-9a04-35dece4180b7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:12.038029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:12.116238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a442d-e086-4b57-b3c4-4264604c9235 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:12.127552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 532/1065] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:21:12.133774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c4f04-24af-4706-9a04-35dece4180b7 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:12.152994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b095a6-38a4-4ef1-b35d-8a090cf86415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:12.153719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b095a6-38a4-4ef1-b35d-8a090cf86415 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:12.154002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b095a6-38a4-4ef1-b35d-8a090cf86415 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:12.154278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b095a6-38a4-4ef1-b35d-8a090cf86415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:12.165959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b095a6-38a4-4ef1-b35d-8a090cf86415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:12.167509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b095a6-38a4-4ef1-b35d-8a090cf86415 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:12.178197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-16b095a6-38a4-4ef1-b35d-8a090cf86415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:21:12.187965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:12.188917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:12.194648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:21:12.194648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:21:12.194648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:12.194648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:12.194648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 534/1066] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:12.194961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b095a6-38a4-4ef1-b35d-8a090cf86415 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:12.248413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c4f04-24af-4706-9a04-35dece4180b7 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:12.277660 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:12.287469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734284ff-a5ec-4d53-a4f8-6e0b45e890da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:12.288332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734284ff-a5ec-4d53-a4f8-6e0b45e890da None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:12.288671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734284ff-a5ec-4d53-a4f8-6e0b45e890da None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:12.289080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734284ff-a5ec-4d53-a4f8-6e0b45e890da None admin] PATH_INFO: `/v3/projects/a38b8c8665424f9b8f9861384ff5c53a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:12.290272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734284ff-a5ec-4d53-a4f8-6e0b45e890da None admin] RBAC: Authorizing `identity:get_project(project_id=a38b8c8665424f9b8f9861384ff5c53a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:12.291718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734284ff-a5ec-4d53-a4f8-6e0b45e890da None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:12.293653 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 533/1067] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:12 2022] GET /identity/v3/projects/a38b8c8665424f9b8f9861384ff5c53a => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:12.300767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734284ff-a5ec-4d53-a4f8-6e0b45e890da None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:12.305277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b095a6-38a4-4ef1-b35d-8a090cf86415 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:12.342080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dd5c6-a788-48e8-8f20-c802f42521b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:12.342761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dd5c6-a788-48e8-8f20-c802f42521b4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:12.343048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dd5c6-a788-48e8-8f20-c802f42521b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:12.343333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dd5c6-a788-48e8-8f20-c802f42521b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:12.344069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dd5c6-a788-48e8-8f20-c802f42521b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:12.344381 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dd5c6-a788-48e8-8f20-c802f42521b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:12.345782 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7dd5c6-a788-48e8-8f20-c802f42521b4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:12.359407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 535/1068] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:12.364664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dd5c6-a788-48e8-8f20-c802f42521b4 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:12.412009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734284ff-a5ec-4d53-a4f8-6e0b45e890da None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:12.423738 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:12.451619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979eba8-e3ac-4653-a1e5-ca29fb31346e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:12.452281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979eba8-e3ac-4653-a1e5-ca29fb31346e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:12.452594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979eba8-e3ac-4653-a1e5-ca29fb31346e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:12.452872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979eba8-e3ac-4653-a1e5-ca29fb31346e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:12.453826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979eba8-e3ac-4653-a1e5-ca29fb31346e None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=a38b8c8665424f9b8f9861384ff5c53a, user.id=6bc11fa221a0403aa9a430034f4359c7) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:12.454019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979eba8-e3ac-4653-a1e5-ca29fb31346e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:12.455469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979eba8-e3ac-4653-a1e5-ca29fb31346e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:12.472509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 534/1069] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:12 2022] GET /identity/v3/role_assignments?user.id=6bc11fa221a0403aa9a430034f4359c7&scope.project.id=a38b8c8665424f9b8f9861384ff5c53a&role.id=f957e09d047b47cfa0891d25b4181263 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:12.483501 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dd5c6-a788-48e8-8f20-c802f42521b4 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:12.523731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dccd73-949a-487e-8756-0c134bb4cc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:12.524581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dccd73-949a-487e-8756-0c134bb4cc10 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:12.524869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dccd73-949a-487e-8756-0c134bb4cc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:12.525145 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dccd73-949a-487e-8756-0c134bb4cc10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:12.527378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:12.535027 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dccd73-949a-487e-8756-0c134bb4cc10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:12.537095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dccd73-949a-487e-8756-0c134bb4cc10 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-c0dccd73-949a-487e-8756-0c134bb4cc10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:21:12.553362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:12.554411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:12.555435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:21:12.555435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:21:12.555435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:21:12.555435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:21:12.555435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:21:12.555435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:21:12.555435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:21:12.555435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:12.556823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 536/1070] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:21:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:12.561044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979eba8-e3ac-4653-a1e5-ca29fb31346e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:12.677419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979eba8-e3ac-4653-a1e5-ca29fb31346e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:12.713438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d92b5-4aed-44ae-87e5-b5bf81e38c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:12.714098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d92b5-4aed-44ae-87e5-b5bf81e38c07 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:12.714378 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d92b5-4aed-44ae-87e5-b5bf81e38c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:12.714652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d92b5-4aed-44ae-87e5-b5bf81e38c07 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:12.715442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d92b5-4aed-44ae-87e5-b5bf81e38c07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:12.715749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d92b5-4aed-44ae-87e5-b5bf81e38c07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:12.717228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d92b5-4aed-44ae-87e5-b5bf81e38c07 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:12.734924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 535/1071] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:21:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:12.736127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dccd73-949a-487e-8756-0c134bb4cc10 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:12.769112 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438ca32-4fc5-47e7-95c7-0185c5551eb7 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:12.769452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438ca32-4fc5-47e7-95c7-0185c5551eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:12.769691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438ca32-4fc5-47e7-95c7-0185c5551eb7 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:12.771228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 536/1072] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:21:12.776420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa406e2b-d682-46de-8d87-b3ab63da9948 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:12.776744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa406e2b-d682-46de-8d87-b3ab63da9948 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:12.777029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa406e2b-d682-46de-8d87-b3ab63da9948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:12.795421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-aa406e2b-d682-46de-8d87-b3ab63da9948 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:12.796665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-aa406e2b-d682-46de-8d87-b3ab63da9948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:12.824977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:12.840431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dccd73-949a-487e-8756-0c134bb4cc10 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:12.872966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 537/1073] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:21:12.876864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e63e37-ab62-4c20-ba32-461c149af502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:12.877778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e63e37-ab62-4c20-ba32-461c149af502 None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:12.878057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e63e37-ab62-4c20-ba32-461c149af502 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:12.878332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e63e37-ab62-4c20-ba32-461c149af502 None admin] PATH_INFO: `/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/c77f6ccc7cb84c04ac532007a04a0f5e/roles/803aed498ea04514b286c6e64ee985cd` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:12.880565 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e63e37-ab62-4c20-ba32-461c149af502 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0638fe2ce7a40299f2d654f58ed535c, user_id=c77f6ccc7cb84c04ac532007a04a0f5e, role_id=803aed498ea04514b286c6e64ee985cd)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:12.882018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e63e37-ab62-4c20-ba32-461c149af502 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:12.895800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa406e2b-d682-46de-8d87-b3ab63da9948 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:12.898126 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 537/1074] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:21:12 2022] PUT /identity/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/c77f6ccc7cb84c04ac532007a04a0f5e/roles/803aed498ea04514b286c6e64ee985cd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:21:13.018494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa406e2b-d682-46de-8d87-b3ab63da9948 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:13.063605 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63572050-e4c9-4694-9b8d-899549f11c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:13.063605 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63572050-e4c9-4694-9b8d-899549f11c16 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:13.063880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63572050-e4c9-4694-9b8d-899549f11c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:13.063979 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63572050-e4c9-4694-9b8d-899549f11c16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:13.098306 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:13.141950 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63572050-e4c9-4694-9b8d-899549f11c16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:13.143690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63572050-e4c9-4694-9b8d-899549f11c16 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:13.255757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 538/1075] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:21:13.261506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e63e37-ab62-4c20-ba32-461c149af502 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:13.319729 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:13.370941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e63e37-ab62-4c20-ba32-461c149af502 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:13.410599 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e0bd0-6bca-4ccc-8384-03477673ac7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:13.411772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e0bd0-6bca-4ccc-8384-03477673ac7b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:13.412289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e0bd0-6bca-4ccc-8384-03477673ac7b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:13.416938 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e0bd0-6bca-4ccc-8384-03477673ac7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:13.430847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e0bd0-6bca-4ccc-8384-03477673ac7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:13.430847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e0bd0-6bca-4ccc-8384-03477673ac7b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-d14e0bd0-6bca-4ccc-8384-03477673ac7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:21:13.450914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:13.452018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:13.453101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:21:13.453101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:21:13.453101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:13.453101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:13.453981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 538/1076] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:13.458346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63572050-e4c9-4694-9b8d-899549f11c16 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:13.527687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:13.572413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63572050-e4c9-4694-9b8d-899549f11c16 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:13.608655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716ba7b-de2e-433b-aebb-55ad535700a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:13.609354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716ba7b-de2e-433b-aebb-55ad535700a6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:13.609643 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716ba7b-de2e-433b-aebb-55ad535700a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:13.609930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716ba7b-de2e-433b-aebb-55ad535700a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:13.610721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716ba7b-de2e-433b-aebb-55ad535700a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:13.611037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716ba7b-de2e-433b-aebb-55ad535700a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:13.612514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716ba7b-de2e-433b-aebb-55ad535700a6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:13.636518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 539/1077] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:14.583636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3695cfe-c8c3-41f5-88df-8643bde0d754 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:14.584207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3695cfe-c8c3-41f5-88df-8643bde0d754 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:14.584207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3695cfe-c8c3-41f5-88df-8643bde0d754 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:14.585581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 539/1078] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:14.590911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ea9ba-4b19-4f0f-9b91-e9877dbf9fb3 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:14.591362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ea9ba-4b19-4f0f-9b91-e9877dbf9fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:14.591608 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ea9ba-4b19-4f0f-9b91-e9877dbf9fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:14.609275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e09ea9ba-4b19-4f0f-9b91-e9877dbf9fb3 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:14.610543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e09ea9ba-4b19-4f0f-9b91-e9877dbf9fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:14.680628 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 540/1079] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:21:14.703660 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3695cfe-c8c3-41f5-88df-8643bde0d754 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:14.806927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3695cfe-c8c3-41f5-88df-8643bde0d754 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:14.814525 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:14.841600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3902a-e38f-4792-92f1-c7ac55d87795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:14.842487 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3902a-e38f-4792-92f1-c7ac55d87795 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:14.842776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3902a-e38f-4792-92f1-c7ac55d87795 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:14.843048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3902a-e38f-4792-92f1-c7ac55d87795 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:14.853826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3902a-e38f-4792-92f1-c7ac55d87795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-8eb3902a-e38f-4792-92f1-c7ac55d87795 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:14.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:14.862516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:14.863551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 540/1080] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:14.869388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ea9ba-4b19-4f0f-9b91-e9877dbf9fb3 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:14.921624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:14.967828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ea9ba-4b19-4f0f-9b91-e9877dbf9fb3 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:15.001488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727d0cd-9d3e-48a7-88fd-3a0bac741497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:15.002151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727d0cd-9d3e-48a7-88fd-3a0bac741497 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:15.002432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727d0cd-9d3e-48a7-88fd-3a0bac741497 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:15.002707 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727d0cd-9d3e-48a7-88fd-3a0bac741497 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:15.003450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727d0cd-9d3e-48a7-88fd-3a0bac741497 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:15.003754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727d0cd-9d3e-48a7-88fd-3a0bac741497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:15.005151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727d0cd-9d3e-48a7-88fd-3a0bac741497 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:15.009459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1f92-ad2a-4399-a1e8-60c7baa3ca59 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:15.009752 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1f92-ad2a-4399-a1e8-60c7baa3ca59 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:15.010033 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1f92-ad2a-4399-a1e8-60c7baa3ca59 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:15.011398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 541/1081] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:15.016515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 541/1082] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:15.016667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdefee-7d22-4686-8f9d-773d36a3ea0a None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:15.016667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdefee-7d22-4686-8f9d-773d36a3ea0a None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:15.016863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdefee-7d22-4686-8f9d-773d36a3ea0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:15.019861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727d0cd-9d3e-48a7-88fd-3a0bac741497 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:15.034477 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7afdefee-7d22-4686-8f9d-773d36a3ea0a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:15.035667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7afdefee-7d22-4686-8f9d-773d36a3ea0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:15.103586 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 542/1083] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:15.119943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727d0cd-9d3e-48a7-88fd-3a0bac741497 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:15.128825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdefee-7d22-4686-8f9d-773d36a3ea0a None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:15.136112 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:15.158295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b327bd-7ee5-4877-89da-10f90a6c36b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:15.159195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b327bd-7ee5-4877-89da-10f90a6c36b8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:15.159475 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b327bd-7ee5-4877-89da-10f90a6c36b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:15.159746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b327bd-7ee5-4877-89da-10f90a6c36b8 None admin] PATH_INFO: `/v3/users/6bc11fa221a0403aa9a430034f4359c7` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:15.160959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b327bd-7ee5-4877-89da-10f90a6c36b8 None admin] RBAC: Authorizing `identity:get_user(user_id=6bc11fa221a0403aa9a430034f4359c7)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:15.162389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b327bd-7ee5-4877-89da-10f90a6c36b8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:15.164705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 542/1084] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:15 2022] GET /identity/v3/users/6bc11fa221a0403aa9a430034f4359c7 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:15.170868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b327bd-7ee5-4877-89da-10f90a6c36b8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:15.225536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:15.232739 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdefee-7d22-4686-8f9d-773d36a3ea0a None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:15.266858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89c8c9-6d0d-44fd-bcb7-1774c1b6e8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:15.267672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89c8c9-6d0d-44fd-bcb7-1774c1b6e8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:15.268212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89c8c9-6d0d-44fd-bcb7-1774c1b6e8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:15.268524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89c8c9-6d0d-44fd-bcb7-1774c1b6e8d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:15.270304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b327bd-7ee5-4877-89da-10f90a6c36b8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:15.279185 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89c8c9-6d0d-44fd-bcb7-1774c1b6e8d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-5a89c8c9-6d0d-44fd-bcb7-1774c1b6e8d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:15.287415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:15.288540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:15.290751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 543/1085] 10.222.0.42 () {56 vars in 1120 bytes} [Thu Jul 21 22:21:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:15.295303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89c8c9-6d0d-44fd-bcb7-1774c1b6e8d7 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:15.305835 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b6145-3e52-49fd-8486-5b522b34c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:15.306692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b6145-3e52-49fd-8486-5b522b34c579 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:15.306970 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b6145-3e52-49fd-8486-5b522b34c579 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:15.307239 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b6145-3e52-49fd-8486-5b522b34c579 None admin] PATH_INFO: `/v3/projects/a38b8c8665424f9b8f9861384ff5c53a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:15.308554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b6145-3e52-49fd-8486-5b522b34c579 None admin] RBAC: Authorizing `identity:get_project(project_id=a38b8c8665424f9b8f9861384ff5c53a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:15.309886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b6145-3e52-49fd-8486-5b522b34c579 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:15.311726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 543/1086] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:15 2022] GET /identity/v3/projects/a38b8c8665424f9b8f9861384ff5c53a => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:15.317630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b6145-3e52-49fd-8486-5b522b34c579 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:15.374833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:15.397375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89c8c9-6d0d-44fd-bcb7-1774c1b6e8d7 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:15.431199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b6145-3e52-49fd-8486-5b522b34c579 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:15.432307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15346ff-3d99-47e6-a2fa-2dc8550ef5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:15.433015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15346ff-3d99-47e6-a2fa-2dc8550ef5ee None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:15.433298 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15346ff-3d99-47e6-a2fa-2dc8550ef5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:15.433576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15346ff-3d99-47e6-a2fa-2dc8550ef5ee None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:15.434354 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15346ff-3d99-47e6-a2fa-2dc8550ef5ee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:15.434661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15346ff-3d99-47e6-a2fa-2dc8550ef5ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:15.436014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15346ff-3d99-47e6-a2fa-2dc8550ef5ee None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:15.449342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 544/1087] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 22:21:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:15.450429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15346ff-3d99-47e6-a2fa-2dc8550ef5ee None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:15.465999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d515b1d-906b-44c9-ad17-e632a4935f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:15.466890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d515b1d-906b-44c9-ad17-e632a4935f6c None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:15.467175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d515b1d-906b-44c9-ad17-e632a4935f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:15.467456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d515b1d-906b-44c9-ad17-e632a4935f6c None admin] PATH_INFO: `/v3/projects/a38b8c8665424f9b8f9861384ff5c53a/users/6bc11fa221a0403aa9a430034f4359c7/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:15.469620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d515b1d-906b-44c9-ad17-e632a4935f6c None admin] RBAC: Authorizing `identity:create_grant(project_id=a38b8c8665424f9b8f9861384ff5c53a, user_id=6bc11fa221a0403aa9a430034f4359c7, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:15.469731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:15.471000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d515b1d-906b-44c9-ad17-e632a4935f6c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:15.496543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 544/1088] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:21:15 2022] PUT /identity/v3/projects/a38b8c8665424f9b8f9861384ff5c53a/users/6bc11fa221a0403aa9a430034f4359c7/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:21:15.558269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15346ff-3d99-47e6-a2fa-2dc8550ef5ee None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:15.596124 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b1381-5937-4873-a79c-c03a0c61d34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:15.596973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b1381-5937-4873-a79c-c03a0c61d34a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:15.597256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b1381-5937-4873-a79c-c03a0c61d34a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:15.597529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b1381-5937-4873-a79c-c03a0c61d34a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:15.610252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b1381-5937-4873-a79c-c03a0c61d34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:15.611752 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b1381-5937-4873-a79c-c03a0c61d34a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-ca0b1381-5937-4873-a79c-c03a0c61d34a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:21:15.630779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:15.631815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:15.633279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:21:15.633279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:21:15.633279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:15.633279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:15.633279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 545/1089] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:15.639150 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d515b1d-906b-44c9-ad17-e632a4935f6c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:15.740094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d515b1d-906b-44c9-ad17-e632a4935f6c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:15.750759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:15.776030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f7abf-4918-423f-8a69-86fe2a63443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:15.776742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f7abf-4918-423f-8a69-86fe2a63443d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:15.777027 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f7abf-4918-423f-8a69-86fe2a63443d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:15.777302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f7abf-4918-423f-8a69-86fe2a63443d None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:15.778026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f7abf-4918-423f-8a69-86fe2a63443d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:15.778333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f7abf-4918-423f-8a69-86fe2a63443d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:15.779704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f7abf-4918-423f-8a69-86fe2a63443d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:15.793010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 545/1090] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:15.798589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b1381-5937-4873-a79c-c03a0c61d34a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:15.868361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:15.900809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b1381-5937-4873-a79c-c03a0c61d34a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:15.935394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af5267-8fbd-4305-bcf7-11e623348e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:15.936189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af5267-8fbd-4305-bcf7-11e623348e8a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:15.936472 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af5267-8fbd-4305-bcf7-11e623348e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:15.936776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af5267-8fbd-4305-bcf7-11e623348e8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:15.944454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af5267-8fbd-4305-bcf7-11e623348e8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:15.945948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af5267-8fbd-4305-bcf7-11e623348e8a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-99af5267-8fbd-4305-bcf7-11e623348e8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:21:15.960047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:15.961737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:15.963120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:15.963120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:15.963120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:21:15.963120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:21:15.963120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:21:15.963120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:21:15.963120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:21:15.963120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:21:15.963120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:21:15.963120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:15.963120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 546/1091] 10.222.0.42 () {56 vars in 1093 bytes} [Thu Jul 21 22:21:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:15.967712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f7abf-4918-423f-8a69-86fe2a63443d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:16.065504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f7abf-4918-423f-8a69-86fe2a63443d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:16.098601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375b95f-9b59-4cd7-aa43-5615887c8579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:16.099267 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375b95f-9b59-4cd7-aa43-5615887c8579 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:16.099545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375b95f-9b59-4cd7-aa43-5615887c8579 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:16.099813 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375b95f-9b59-4cd7-aa43-5615887c8579 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:16.100684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375b95f-9b59-4cd7-aa43-5615887c8579 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:16.100991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375b95f-9b59-4cd7-aa43-5615887c8579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:16.103099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375b95f-9b59-4cd7-aa43-5615887c8579 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:16.121618 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 546/1092] 10.222.0.42 () {56 vars in 1133 bytes} [Thu Jul 21 22:21:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:16.127173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af5267-8fbd-4305-bcf7-11e623348e8a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:16.228856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af5267-8fbd-4305-bcf7-11e623348e8a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:16.257191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:16.263714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6a5a1-b4b9-4897-afce-135e26984e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:16.264382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6a5a1-b4b9-4897-afce-135e26984e2e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:16.264711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6a5a1-b4b9-4897-afce-135e26984e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:16.264994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6a5a1-b4b9-4897-afce-135e26984e2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:16.334124 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6a5a1-b4b9-4897-afce-135e26984e2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:16.335483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6a5a1-b4b9-4897-afce-135e26984e2e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:16.440903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 547/1093] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:21:16.446979 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375b95f-9b59-4cd7-aa43-5615887c8579 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:16.474548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:16.550088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375b95f-9b59-4cd7-aa43-5615887c8579 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:16.585476 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b9611-b7a4-4a56-9586-d55689fae46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:16.586799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b9611-b7a4-4a56-9586-d55689fae46d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:16.587097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b9611-b7a4-4a56-9586-d55689fae46d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:16.587445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b9611-b7a4-4a56-9586-d55689fae46d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:16.599743 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b9611-b7a4-4a56-9586-d55689fae46d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:16.601315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b9611-b7a4-4a56-9586-d55689fae46d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-078b9611-b7a4-4a56-9586-d55689fae46d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:21:16.621107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:16.622313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:16.623358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:21:16.623358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:21:16.623358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:21:16.623358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:16.623358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 547/1094] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:16.628099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6a5a1-b4b9-4897-afce-135e26984e2e None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:16.660371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:16.734283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6a5a1-b4b9-4897-afce-135e26984e2e None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:16.769041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bb2dd-a70f-4c97-830c-027f30dfa2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:16.769713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bb2dd-a70f-4c97-830c-027f30dfa2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:16.769994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bb2dd-a70f-4c97-830c-027f30dfa2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:16.770269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bb2dd-a70f-4c97-830c-027f30dfa2c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:16.770999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bb2dd-a70f-4c97-830c-027f30dfa2c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:16.771306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bb2dd-a70f-4c97-830c-027f30dfa2c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:16.772698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bb2dd-a70f-4c97-830c-027f30dfa2c9 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:16.784177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 548/1095] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:16.790085 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b9611-b7a4-4a56-9586-d55689fae46d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:16.879007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:16.905693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b9611-b7a4-4a56-9586-d55689fae46d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:16.940170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09912b1-ff72-4e2b-b6a4-aacdb76060fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:16.941000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09912b1-ff72-4e2b-b6a4-aacdb76060fa None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:16.941286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09912b1-ff72-4e2b-b6a4-aacdb76060fa None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:16.941562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09912b1-ff72-4e2b-b6a4-aacdb76060fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:16.949757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09912b1-ff72-4e2b-b6a4-aacdb76060fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:16.951216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09912b1-ff72-4e2b-b6a4-aacdb76060fa None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b09912b1-ff72-4e2b-b6a4-aacdb76060fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:21:16.965630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:16.966731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:16.967787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:21:16.967787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:21:16.967787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:21:16.967787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:21:16.967787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:21:16.967787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:21:16.967787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:21:16.967787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:16.967787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 548/1096] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:21:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:21:16.973176 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bb2dd-a70f-4c97-830c-027f30dfa2c9 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:17.068664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:17.080072 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bb2dd-a70f-4c97-830c-027f30dfa2c9 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:17.120344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f8b08-2873-48c0-834d-40652481c69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:17.121053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f8b08-2873-48c0-834d-40652481c69a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:17.121340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f8b08-2873-48c0-834d-40652481c69a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:17.121615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f8b08-2873-48c0-834d-40652481c69a None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:17.122448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f8b08-2873-48c0-834d-40652481c69a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:17.122759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f8b08-2873-48c0-834d-40652481c69a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:17.124222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f8b08-2873-48c0-834d-40652481c69a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:17.137837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 549/1097] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:21:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:17.144442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09912b1-ff72-4e2b-b6a4-aacdb76060fa None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:17.250508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09912b1-ff72-4e2b-b6a4-aacdb76060fa None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:17.284361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8fc39-77af-42e8-9f69-d170bede0018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:17.285046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8fc39-77af-42e8-9f69-d170bede0018 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:17.285331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8fc39-77af-42e8-9f69-d170bede0018 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:17.285607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8fc39-77af-42e8-9f69-d170bede0018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:17.286442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8fc39-77af-42e8-9f69-d170bede0018 None admin] RBAC: Adding query filter params (role.id=803aed498ea04514b286c6e64ee985cd, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=c77f6ccc7cb84c04ac532007a04a0f5e) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:17.286757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8fc39-77af-42e8-9f69-d170bede0018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:17.288193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8fc39-77af-42e8-9f69-d170bede0018 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:17.294917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 549/1098] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:17 2022] GET /identity/v3/role_assignments?user.id=c77f6ccc7cb84c04ac532007a04a0f5e&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=803aed498ea04514b286c6e64ee985cd => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:17.563045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd7765-b589-4684-be67-7c20d414b732 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:17.563334 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd7765-b589-4684-be67-7c20d414b732 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:17.563611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd7765-b589-4684-be67-7c20d414b732 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:17.565056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 550/1099] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:17.570674 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198dd0ad-de34-440f-82ce-90ca1b147866 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:17.570971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198dd0ad-de34-440f-82ce-90ca1b147866 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:17.571251 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198dd0ad-de34-440f-82ce-90ca1b147866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:17.577358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:17.590413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-198dd0ad-de34-440f-82ce-90ca1b147866 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:17.591621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-198dd0ad-de34-440f-82ce-90ca1b147866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:17.662075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 550/1100] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:17.685729 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd7765-b589-4684-be67-7c20d414b732 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:17.791014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd7765-b589-4684-be67-7c20d414b732 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:17.825920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f742d2-faa3-453b-89d8-4e7dd7f982dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:17.826731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f742d2-faa3-453b-89d8-4e7dd7f982dd None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:17.827016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f742d2-faa3-453b-89d8-4e7dd7f982dd None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:17.827293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f742d2-faa3-453b-89d8-4e7dd7f982dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:17.838093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f742d2-faa3-453b-89d8-4e7dd7f982dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-17f742d2-faa3-453b-89d8-4e7dd7f982dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:17.845795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:17.846834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:17.847710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 551/1101] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:17.854195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198dd0ad-de34-440f-82ce-90ca1b147866 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:17.957669 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198dd0ad-de34-440f-82ce-90ca1b147866 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:17.992283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbf79a-fc97-4d78-b60f-157cda545e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:17.993082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbf79a-fc97-4d78-b60f-157cda545e00 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:17.993361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbf79a-fc97-4d78-b60f-157cda545e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:17.993631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbf79a-fc97-4d78-b60f-157cda545e00 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:17.994379 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbf79a-fc97-4d78-b60f-157cda545e00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:17.994679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbf79a-fc97-4d78-b60f-157cda545e00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:17.996056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbf79a-fc97-4d78-b60f-157cda545e00 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:18.001510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:18.005258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 551/1102] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:18.011335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f742d2-faa3-453b-89d8-4e7dd7f982dd None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:18.024706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:18.116050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f742d2-faa3-453b-89d8-4e7dd7f982dd None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:18.155357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5948983-d7d2-4fd7-9860-fe010afdc210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:18.156242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5948983-d7d2-4fd7-9860-fe010afdc210 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:18.156552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5948983-d7d2-4fd7-9860-fe010afdc210 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:18.156831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5948983-d7d2-4fd7-9860-fe010afdc210 None admin] PATH_INFO: `/v3/users/6bc11fa221a0403aa9a430034f4359c7` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:18.158041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5948983-d7d2-4fd7-9860-fe010afdc210 None admin] RBAC: Authorizing `identity:get_user(user_id=6bc11fa221a0403aa9a430034f4359c7)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:18.159474 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5948983-d7d2-4fd7-9860-fe010afdc210 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:18.161149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 552/1103] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:18 2022] GET /identity/v3/users/6bc11fa221a0403aa9a430034f4359c7 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:18.167561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbf79a-fc97-4d78-b60f-157cda545e00 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:18.269077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbf79a-fc97-4d78-b60f-157cda545e00 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:18.301451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd17935-76c8-42a2-9294-100f99ff45c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:18.302092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd17935-76c8-42a2-9294-100f99ff45c3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:18.302375 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd17935-76c8-42a2-9294-100f99ff45c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:18.302649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd17935-76c8-42a2-9294-100f99ff45c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:18.362234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:18.369454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd17935-76c8-42a2-9294-100f99ff45c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:18.370902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd17935-76c8-42a2-9294-100f99ff45c3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:18.469182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 552/1104] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:21:18.475105 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5948983-d7d2-4fd7-9860-fe010afdc210 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:18.552236 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:18.575380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5948983-d7d2-4fd7-9860-fe010afdc210 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:18.608238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1586141-c363-4105-bdce-8a683f758c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:18.609158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1586141-c363-4105-bdce-8a683f758c09 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:18.609446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1586141-c363-4105-bdce-8a683f758c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:18.609757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1586141-c363-4105-bdce-8a683f758c09 None admin] PATH_INFO: `/v3/projects/a38b8c8665424f9b8f9861384ff5c53a` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:18.610943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1586141-c363-4105-bdce-8a683f758c09 None admin] RBAC: Authorizing `identity:get_project(project_id=a38b8c8665424f9b8f9861384ff5c53a)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:18.612378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1586141-c363-4105-bdce-8a683f758c09 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:18.614134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 553/1105] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:18 2022] GET /identity/v3/projects/a38b8c8665424f9b8f9861384ff5c53a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:18.621091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd17935-76c8-42a2-9294-100f99ff45c3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:18.712542 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:18.718984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd17935-76c8-42a2-9294-100f99ff45c3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:18.754428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda5aaf-9625-4101-9193-9238634bd254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:18.755082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda5aaf-9625-4101-9193-9238634bd254 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:18.755362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda5aaf-9625-4101-9193-9238634bd254 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:18.755634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda5aaf-9625-4101-9193-9238634bd254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:18.756464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda5aaf-9625-4101-9193-9238634bd254 None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=a38b8c8665424f9b8f9861384ff5c53a, user.id=6bc11fa221a0403aa9a430034f4359c7) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:18.756803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda5aaf-9625-4101-9193-9238634bd254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:18.758244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda5aaf-9625-4101-9193-9238634bd254 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:18.764071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 553/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:18 2022] GET /identity/v3/role_assignments?user.id=6bc11fa221a0403aa9a430034f4359c7&scope.project.id=a38b8c8665424f9b8f9861384ff5c53a&role.id=f957e09d047b47cfa0891d25b4181263 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:20.873691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b394296-0d09-4383-8438-573c5dc0b173 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:20.874279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b394296-0d09-4383-8438-573c5dc0b173 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:20.874279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b394296-0d09-4383-8438-573c5dc0b173 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:20.875597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 554/1107] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:20.880892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8756dff-12e2-4dd8-b4de-43b5165f222f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:20.881281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8756dff-12e2-4dd8-b4de-43b5165f222f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:20.881450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8756dff-12e2-4dd8-b4de-43b5165f222f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:20.900575 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a8756dff-12e2-4dd8-b4de-43b5165f222f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:20.901766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a8756dff-12e2-4dd8-b4de-43b5165f222f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:20.968458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 554/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:20.994261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b394296-0d09-4383-8438-573c5dc0b173 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:21.094702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b394296-0d09-4383-8438-573c5dc0b173 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:21.129671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228983f-b5bd-4d21-89a7-09ac3ec54c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:21.130392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228983f-b5bd-4d21-89a7-09ac3ec54c31 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:21.130674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228983f-b5bd-4d21-89a7-09ac3ec54c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:21.130949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228983f-b5bd-4d21-89a7-09ac3ec54c31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:21.132634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228983f-b5bd-4d21-89a7-09ac3ec54c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:21.134090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228983f-b5bd-4d21-89a7-09ac3ec54c31 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:21.135712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 555/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:21.141678 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8756dff-12e2-4dd8-b4de-43b5165f222f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:21.231415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:21.242360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8756dff-12e2-4dd8-b4de-43b5165f222f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:21.276175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf65fc8-1a26-4e62-af84-4e3e076f5fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:21.276874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf65fc8-1a26-4e62-af84-4e3e076f5fc0 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:21.277161 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf65fc8-1a26-4e62-af84-4e3e076f5fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:21.277438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf65fc8-1a26-4e62-af84-4e3e076f5fc0 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:21.278144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf65fc8-1a26-4e62-af84-4e3e076f5fc0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:21.279652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf65fc8-1a26-4e62-af84-4e3e076f5fc0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:21.298982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 555/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:21:21 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:21:23.390636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5150c8-0a13-40f1-92d4-966b51d4c295 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:23.391201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5150c8-0a13-40f1-92d4-966b51d4c295 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:23.391201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5150c8-0a13-40f1-92d4-966b51d4c295 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:23.392566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 556/1111] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:23.398820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a3fd8-4399-4f57-9fbe-772aa30e644e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:23.399249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a3fd8-4399-4f57-9fbe-772aa30e644e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:23.399351 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a3fd8-4399-4f57-9fbe-772aa30e644e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:23.416403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e22a3fd8-4399-4f57-9fbe-772aa30e644e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:23.417583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e22a3fd8-4399-4f57-9fbe-772aa30e644e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:23.483507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 556/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:23.507287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5150c8-0a13-40f1-92d4-966b51d4c295 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:23.523444 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:23.609922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5150c8-0a13-40f1-92d4-966b51d4c295 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:23.642374 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a0c8c-eea3-44fe-85d1-2244d1576b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:23.643407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a0c8c-eea3-44fe-85d1-2244d1576b47 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:23.643686 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a0c8c-eea3-44fe-85d1-2244d1576b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:23.643957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a0c8c-eea3-44fe-85d1-2244d1576b47 None admin] PATH_INFO: `/v3/roles/80eea410deca46e2924f27afe5136d55` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:23.645435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a0c8c-eea3-44fe-85d1-2244d1576b47 None admin] RBAC: Authorizing `identity:get_role(role_id=80eea410deca46e2924f27afe5136d55)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:23.646793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a0c8c-eea3-44fe-85d1-2244d1576b47 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:23.648698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 557/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:23 2022] GET /identity/v3/roles/80eea410deca46e2924f27afe5136d55 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:23.654599 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a3fd8-4399-4f57-9fbe-772aa30e644e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:23.729194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:23.752811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a3fd8-4399-4f57-9fbe-772aa30e644e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:23.786289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b610349-a2a7-46f6-8e69-013b8ed3ac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:23.787143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b610349-a2a7-46f6-8e69-013b8ed3ac75 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:23.787422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b610349-a2a7-46f6-8e69-013b8ed3ac75 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:23.787692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b610349-a2a7-46f6-8e69-013b8ed3ac75 None admin] PATH_INFO: `/v3/users/f0cc82f8fb904110a0b0b02ae8be0e2d` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:23.801142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b610349-a2a7-46f6-8e69-013b8ed3ac75 None admin] RBAC: Authorizing `identity:get_user(user_id=f0cc82f8fb904110a0b0b02ae8be0e2d)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:23.802577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b610349-a2a7-46f6-8e69-013b8ed3ac75 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:23.804741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 557/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:23 2022] GET /identity/v3/users/f0cc82f8fb904110a0b0b02ae8be0e2d => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:23.810365 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a0c8c-eea3-44fe-85d1-2244d1576b47 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:23.895778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:23.914659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a0c8c-eea3-44fe-85d1-2244d1576b47 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:23.948412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb9e57-3e83-4a1d-b8af-7828676d9061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:23.949107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb9e57-3e83-4a1d-b8af-7828676d9061 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:23.949395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb9e57-3e83-4a1d-b8af-7828676d9061 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:23.949675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb9e57-3e83-4a1d-b8af-7828676d9061 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:24.019188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb9e57-3e83-4a1d-b8af-7828676d9061 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:24.020672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb9e57-3e83-4a1d-b8af-7828676d9061 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:24.117640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:24.137558 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 558/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:21:24.143728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b610349-a2a7-46f6-8e69-013b8ed3ac75 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:24.245859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b610349-a2a7-46f6-8e69-013b8ed3ac75 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:24.280361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b7249-c493-4a96-8e2c-ac361a639ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:24.281247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b7249-c493-4a96-8e2c-ac361a639ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:24.281526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b7249-c493-4a96-8e2c-ac361a639ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:24.281796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b7249-c493-4a96-8e2c-ac361a639ca0 None admin] PATH_INFO: `/v3/projects/a38b8c8665424f9b8f9861384ff5c53a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:24.282968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b7249-c493-4a96-8e2c-ac361a639ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=a38b8c8665424f9b8f9861384ff5c53a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:24.284400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b7249-c493-4a96-8e2c-ac361a639ca0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:24.286134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 558/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:24 2022] GET /identity/v3/projects/a38b8c8665424f9b8f9861384ff5c53a => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:24.292728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb9e57-3e83-4a1d-b8af-7828676d9061 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:24.391231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb9e57-3e83-4a1d-b8af-7828676d9061 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:24.425448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff950d7-9cfc-4e69-9e7c-7e695971bdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:24.426142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff950d7-9cfc-4e69-9e7c-7e695971bdaf None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:24.426442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff950d7-9cfc-4e69-9e7c-7e695971bdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:24.426736 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff950d7-9cfc-4e69-9e7c-7e695971bdaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:24.433188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff950d7-9cfc-4e69-9e7c-7e695971bdaf None admin] RBAC: Adding query filter params (role.id=80eea410deca46e2924f27afe5136d55, scope.project.id=a38b8c8665424f9b8f9861384ff5c53a, user.id=f0cc82f8fb904110a0b0b02ae8be0e2d) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:24.433188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff950d7-9cfc-4e69-9e7c-7e695971bdaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:24.433188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff950d7-9cfc-4e69-9e7c-7e695971bdaf None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:24.443860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 559/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:24 2022] GET /identity/v3/role_assignments?user.id=f0cc82f8fb904110a0b0b02ae8be0e2d&scope.project.id=a38b8c8665424f9b8f9861384ff5c53a&role.id=80eea410deca46e2924f27afe5136d55 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:26.505753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e5f55-b406-4c1f-a4c8-6b455458f78f None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:26.506330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e5f55-b406-4c1f-a4c8-6b455458f78f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:26.506330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e5f55-b406-4c1f-a4c8-6b455458f78f None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:26.507681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 559/1118] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:26.513708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d0549-4707-4d55-a1c7-5d93b99fdb14 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:26.514121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d0549-4707-4d55-a1c7-5d93b99fdb14 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:26.514272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d0549-4707-4d55-a1c7-5d93b99fdb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:26.531106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-670d0549-4707-4d55-a1c7-5d93b99fdb14 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:26.532305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-670d0549-4707-4d55-a1c7-5d93b99fdb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:26.574889 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:26.604235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 560/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:26.628300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e5f55-b406-4c1f-a4c8-6b455458f78f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:26.633887 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:26.744278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e5f55-b406-4c1f-a4c8-6b455458f78f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:26.783911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157140da-68f6-470a-9858-46ed9b2d612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:26.785240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157140da-68f6-470a-9858-46ed9b2d612f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:26.785527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157140da-68f6-470a-9858-46ed9b2d612f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:26.785928 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157140da-68f6-470a-9858-46ed9b2d612f None admin] PATH_INFO: `/v3/roles/80eea410deca46e2924f27afe5136d55` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:26.787472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157140da-68f6-470a-9858-46ed9b2d612f None admin] RBAC: Authorizing `identity:get_role(role_id=80eea410deca46e2924f27afe5136d55)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:26.789942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157140da-68f6-470a-9858-46ed9b2d612f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:26.791360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 560/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:26 2022] GET /identity/v3/roles/80eea410deca46e2924f27afe5136d55 => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:26.797777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d0549-4707-4d55-a1c7-5d93b99fdb14 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:26.921111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d0549-4707-4d55-a1c7-5d93b99fdb14 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:26.961919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffacf6-0f17-4b39-912f-dd3fdab8174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:26.962801 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffacf6-0f17-4b39-912f-dd3fdab8174a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:26.963086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffacf6-0f17-4b39-912f-dd3fdab8174a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:26.963364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffacf6-0f17-4b39-912f-dd3fdab8174a None admin] PATH_INFO: `/v3/users/f0cc82f8fb904110a0b0b02ae8be0e2d` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:26.964624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffacf6-0f17-4b39-912f-dd3fdab8174a None admin] RBAC: Authorizing `identity:get_user(user_id=f0cc82f8fb904110a0b0b02ae8be0e2d)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:26.966064 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffacf6-0f17-4b39-912f-dd3fdab8174a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:26.967882 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 561/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:26 2022] GET /identity/v3/users/f0cc82f8fb904110a0b0b02ae8be0e2d => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:26.978262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157140da-68f6-470a-9858-46ed9b2d612f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:27.051850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:27.085711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157140da-68f6-470a-9858-46ed9b2d612f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:27.123468 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c73a75-89f1-45c3-bf17-1d10416626be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:27.124326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c73a75-89f1-45c3-bf17-1d10416626be None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:27.124631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c73a75-89f1-45c3-bf17-1d10416626be None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:27.124905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c73a75-89f1-45c3-bf17-1d10416626be None admin] PATH_INFO: `/v3/projects/a38b8c8665424f9b8f9861384ff5c53a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:27.126108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c73a75-89f1-45c3-bf17-1d10416626be None admin] RBAC: Authorizing `identity:get_project(project_id=a38b8c8665424f9b8f9861384ff5c53a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:27.127542 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c73a75-89f1-45c3-bf17-1d10416626be None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:27.129293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 561/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:26 2022] GET /identity/v3/projects/a38b8c8665424f9b8f9861384ff5c53a => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:27.135734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffacf6-0f17-4b39-912f-dd3fdab8174a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:27.187680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:27.253474 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffacf6-0f17-4b39-912f-dd3fdab8174a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:27.294970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57f167-252d-4c92-a66d-d569cef5b698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:27.296041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57f167-252d-4c92-a66d-d569cef5b698 None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:27.296647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57f167-252d-4c92-a66d-d569cef5b698 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:27.297151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57f167-252d-4c92-a66d-d569cef5b698 None admin] PATH_INFO: `/v3/projects/a38b8c8665424f9b8f9861384ff5c53a/users/f0cc82f8fb904110a0b0b02ae8be0e2d/roles/80eea410deca46e2924f27afe5136d55` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:27.300007 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57f167-252d-4c92-a66d-d569cef5b698 None admin] RBAC: Authorizing `identity:create_grant(project_id=a38b8c8665424f9b8f9861384ff5c53a, user_id=f0cc82f8fb904110a0b0b02ae8be0e2d, role_id=80eea410deca46e2924f27afe5136d55)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:27.302681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57f167-252d-4c92-a66d-d569cef5b698 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:27.319203 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 562/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:21:27 2022] PUT /identity/v3/projects/a38b8c8665424f9b8f9861384ff5c53a/users/f0cc82f8fb904110a0b0b02ae8be0e2d/roles/80eea410deca46e2924f27afe5136d55 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:21:29.373588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01570ff-e880-4507-a3c6-0a54dc2708d9 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:29.374172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01570ff-e880-4507-a3c6-0a54dc2708d9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:29.374172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01570ff-e880-4507-a3c6-0a54dc2708d9 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:29.375479 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 562/1124] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:29.380826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8826c6-5456-4072-9950-69bd287f8271 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:29.381202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8826c6-5456-4072-9950-69bd287f8271 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:29.381389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8826c6-5456-4072-9950-69bd287f8271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:29.399169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-cb8826c6-5456-4072-9950-69bd287f8271 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:29.400357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-cb8826c6-5456-4072-9950-69bd287f8271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:29.469227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 563/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:29.493253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01570ff-e880-4507-a3c6-0a54dc2708d9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:29.593102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01570ff-e880-4507-a3c6-0a54dc2708d9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:29.626282 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753fc7c-6516-4dcf-95c2-da19e55932e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:29.627047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753fc7c-6516-4dcf-95c2-da19e55932e1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:29.627328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753fc7c-6516-4dcf-95c2-da19e55932e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:29.627599 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753fc7c-6516-4dcf-95c2-da19e55932e1 None admin] PATH_INFO: `/v3/roles/80eea410deca46e2924f27afe5136d55` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:29.629084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753fc7c-6516-4dcf-95c2-da19e55932e1 None admin] RBAC: Authorizing `identity:get_role(role_id=80eea410deca46e2924f27afe5136d55)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:29.630442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753fc7c-6516-4dcf-95c2-da19e55932e1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:29.631886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 563/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:29 2022] GET /identity/v3/roles/80eea410deca46e2924f27afe5136d55 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:29.637774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8826c6-5456-4072-9950-69bd287f8271 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:29.671429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:29.738851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8826c6-5456-4072-9950-69bd287f8271 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:29.771691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20b4a0-83ea-4843-a0de-32b4b51a1c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:29.773112 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b4a0-83ea-4843-a0de-32b4b51a1c88 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:29.773401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b4a0-83ea-4843-a0de-32b4b51a1c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:29.773674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b4a0-83ea-4843-a0de-32b4b51a1c88 None admin] PATH_INFO: `/v3/users/f0cc82f8fb904110a0b0b02ae8be0e2d` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:29.774881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20b4a0-83ea-4843-a0de-32b4b51a1c88 None admin] RBAC: Authorizing `identity:get_user(user_id=f0cc82f8fb904110a0b0b02ae8be0e2d)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:29.776328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20b4a0-83ea-4843-a0de-32b4b51a1c88 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:29.778009 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 564/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:29 2022] GET /identity/v3/users/f0cc82f8fb904110a0b0b02ae8be0e2d => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:29.783859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753fc7c-6516-4dcf-95c2-da19e55932e1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:29.793320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:29.885074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753fc7c-6516-4dcf-95c2-da19e55932e1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:29.923258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037903f-4d54-455c-b00a-821d15978c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:29.923904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037903f-4d54-455c-b00a-821d15978c2d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:29.924191 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037903f-4d54-455c-b00a-821d15978c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:29.924471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037903f-4d54-455c-b00a-821d15978c2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:29.992508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037903f-4d54-455c-b00a-821d15978c2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:29.993982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037903f-4d54-455c-b00a-821d15978c2d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:30.001570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:30.092201 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 564/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:21:30.098306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20b4a0-83ea-4843-a0de-32b4b51a1c88 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:30.203952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20b4a0-83ea-4843-a0de-32b4b51a1c88 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:30.205470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:30.239508 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb123e6-f9ec-42aa-ae92-94740913940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:30.240372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb123e6-f9ec-42aa-ae92-94740913940b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:30.240684 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb123e6-f9ec-42aa-ae92-94740913940b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:30.241500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb123e6-f9ec-42aa-ae92-94740913940b None admin] PATH_INFO: `/v3/projects/a38b8c8665424f9b8f9861384ff5c53a` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:30.243103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb123e6-f9ec-42aa-ae92-94740913940b None admin] RBAC: Authorizing `identity:get_project(project_id=a38b8c8665424f9b8f9861384ff5c53a)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:30.244577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb123e6-f9ec-42aa-ae92-94740913940b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:30.246287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 565/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:30 2022] GET /identity/v3/projects/a38b8c8665424f9b8f9861384ff5c53a => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:30.252842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037903f-4d54-455c-b00a-821d15978c2d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:30.352256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037903f-4d54-455c-b00a-821d15978c2d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:30.360039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:30.389165 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2c22d-d0f8-4364-a816-7f9e6db4d182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:30.389812 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2c22d-d0f8-4364-a816-7f9e6db4d182 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:30.390089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2c22d-d0f8-4364-a816-7f9e6db4d182 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:30.390373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2c22d-d0f8-4364-a816-7f9e6db4d182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:30.391207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2c22d-d0f8-4364-a816-7f9e6db4d182 None admin] RBAC: Adding query filter params (role.id=80eea410deca46e2924f27afe5136d55, scope.project.id=a38b8c8665424f9b8f9861384ff5c53a, user.id=f0cc82f8fb904110a0b0b02ae8be0e2d) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:30.391526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2c22d-d0f8-4364-a816-7f9e6db4d182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:30.393274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2c22d-d0f8-4364-a816-7f9e6db4d182 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:30.399320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 565/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:30 2022] GET /identity/v3/role_assignments?user.id=f0cc82f8fb904110a0b0b02ae8be0e2d&scope.project.id=a38b8c8665424f9b8f9861384ff5c53a&role.id=80eea410deca46e2924f27afe5136d55 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:32.459377 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385e747-e102-43ec-b71d-a0fc647fbab2 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:32.459902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385e747-e102-43ec-b71d-a0fc647fbab2 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:32.460061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385e747-e102-43ec-b71d-a0fc647fbab2 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:32.461469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 566/1131] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:32.466739 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f47d58-7b24-4726-8f79-9e71d57a63d7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:32.467179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f47d58-7b24-4726-8f79-9e71d57a63d7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:32.467280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f47d58-7b24-4726-8f79-9e71d57a63d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:32.483754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-52f47d58-7b24-4726-8f79-9e71d57a63d7 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:32.484996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-52f47d58-7b24-4726-8f79-9e71d57a63d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:32.552037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 566/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:32.576153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385e747-e102-43ec-b71d-a0fc647fbab2 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:32.676196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385e747-e102-43ec-b71d-a0fc647fbab2 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:32.710638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946529e-6a39-409a-b4cd-8cb803b35a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:32.711349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946529e-6a39-409a-b4cd-8cb803b35a29 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:32.711635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946529e-6a39-409a-b4cd-8cb803b35a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:32.711913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946529e-6a39-409a-b4cd-8cb803b35a29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:32.713124 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946529e-6a39-409a-b4cd-8cb803b35a29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:32.714563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946529e-6a39-409a-b4cd-8cb803b35a29 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:32.716190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 567/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:32.722117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f47d58-7b24-4726-8f79-9e71d57a63d7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:32.818258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:32.821805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f47d58-7b24-4726-8f79-9e71d57a63d7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:32.855685 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c6c5e-2520-4723-953c-b03e7e8d5a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:32.856337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c6c5e-2520-4723-953c-b03e7e8d5a40 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:32.856649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c6c5e-2520-4723-953c-b03e7e8d5a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:32.856924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c6c5e-2520-4723-953c-b03e7e8d5a40 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:32.857625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c6c5e-2520-4723-953c-b03e7e8d5a40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:32.859053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c6c5e-2520-4723-953c-b03e7e8d5a40 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:32.885213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 567/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:21:32 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:21:34.955989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491d8d-31d4-4b7c-aad0-8e2c70edb61e None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:34.956548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491d8d-31d4-4b7c-aad0-8e2c70edb61e None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:34.956665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491d8d-31d4-4b7c-aad0-8e2c70edb61e None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:34.957939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 568/1135] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:34.963500 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce3d7d-2e65-4b8b-a58f-b73504270c1a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:34.963899 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce3d7d-2e65-4b8b-a58f-b73504270c1a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:34.964063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce3d7d-2e65-4b8b-a58f-b73504270c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:34.980807 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-75ce3d7d-2e65-4b8b-a58f-b73504270c1a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:34.982007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-75ce3d7d-2e65-4b8b-a58f-b73504270c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:35.049931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 568/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:35.075567 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491d8d-31d4-4b7c-aad0-8e2c70edb61e None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:35.117658 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:35.184242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491d8d-31d4-4b7c-aad0-8e2c70edb61e None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:35.220077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07457c-c95d-4ef0-9c1a-9cb6a5b78e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:35.220807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07457c-c95d-4ef0-9c1a-9cb6a5b78e7f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:35.221090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07457c-c95d-4ef0-9c1a-9cb6a5b78e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:35.221369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07457c-c95d-4ef0-9c1a-9cb6a5b78e7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:35.222534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07457c-c95d-4ef0-9c1a-9cb6a5b78e7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:35.223956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07457c-c95d-4ef0-9c1a-9cb6a5b78e7f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:35.225662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 569/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:21:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:35.231504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce3d7d-2e65-4b8b-a58f-b73504270c1a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:35.313819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:35.333413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce3d7d-2e65-4b8b-a58f-b73504270c1a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:35.370825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327960e-df06-4f87-bf1d-f70ad637bf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:35.371494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327960e-df06-4f87-bf1d-f70ad637bf18 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:35.371776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327960e-df06-4f87-bf1d-f70ad637bf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:35.372051 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327960e-df06-4f87-bf1d-f70ad637bf18 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:35.372799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327960e-df06-4f87-bf1d-f70ad637bf18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:35.374255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327960e-df06-4f87-bf1d-f70ad637bf18 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:35.395303 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 569/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:21:35 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:21:37.534751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d818b4-72de-4803-b733-88617121ba2d None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:37.535303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d818b4-72de-4803-b733-88617121ba2d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:37.535303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d818b4-72de-4803-b733-88617121ba2d None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:37.536704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 570/1139] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:37.542617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1115c-b5ec-4675-add4-e8b9bdc1b26b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:37.543084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1115c-b5ec-4675-add4-e8b9bdc1b26b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:37.543201 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1115c-b5ec-4675-add4-e8b9bdc1b26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:37.562339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c7b1115c-b5ec-4675-add4-e8b9bdc1b26b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:37.563641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c7b1115c-b5ec-4675-add4-e8b9bdc1b26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:37.640629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 570/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:37.666233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d818b4-72de-4803-b733-88617121ba2d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:37.782434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d818b4-72de-4803-b733-88617121ba2d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:37.791474 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:37.821248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23881f9b-d4c3-4a39-9e9b-2948a7f80e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:37.822227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23881f9b-d4c3-4a39-9e9b-2948a7f80e6f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:37.822546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23881f9b-d4c3-4a39-9e9b-2948a7f80e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:37.822889 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23881f9b-d4c3-4a39-9e9b-2948a7f80e6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:37.835901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23881f9b-d4c3-4a39-9e9b-2948a7f80e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-23881f9b-d4c3-4a39-9e9b-2948a7f80e6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:37.844468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:37.845694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:37.846724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 571/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:37.855441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1115c-b5ec-4675-add4-e8b9bdc1b26b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:37.937954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:37.978713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1115c-b5ec-4675-add4-e8b9bdc1b26b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:38.020292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201465a-320e-4883-9afb-5ca2dd5d1eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:38.021038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201465a-320e-4883-9afb-5ca2dd5d1eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:38.021315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201465a-320e-4883-9afb-5ca2dd5d1eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:38.022208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201465a-320e-4883-9afb-5ca2dd5d1eb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:38.022993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201465a-320e-4883-9afb-5ca2dd5d1eb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:38.023293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201465a-320e-4883-9afb-5ca2dd5d1eb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:38.024787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201465a-320e-4883-9afb-5ca2dd5d1eb6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:38.034514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 571/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:38.041783 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23881f9b-d4c3-4a39-9e9b-2948a7f80e6f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:38.176929 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23881f9b-d4c3-4a39-9e9b-2948a7f80e6f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:38.218476 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6de304-3563-43f6-b711-7625333001ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:38.219677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6de304-3563-43f6-b711-7625333001ec None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:38.220042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6de304-3563-43f6-b711-7625333001ec None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:38.220325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6de304-3563-43f6-b711-7625333001ec None admin] PATH_INFO: `/v3/users/eedc60e6e4fd47938e4bc6280425d17c` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:38.236623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6de304-3563-43f6-b711-7625333001ec None admin] RBAC: Authorizing `identity:get_user(user_id=eedc60e6e4fd47938e4bc6280425d17c)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:38.238249 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6de304-3563-43f6-b711-7625333001ec None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:38.240232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 572/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:38 2022] GET /identity/v3/users/eedc60e6e4fd47938e4bc6280425d17c => generated 286 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:38.246550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201465a-320e-4883-9afb-5ca2dd5d1eb6 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:38.385085 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201465a-320e-4883-9afb-5ca2dd5d1eb6 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:38.423030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:38.430492 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b36119-1bf9-4653-b746-aeb0214f142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:38.431151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b36119-1bf9-4653-b746-aeb0214f142f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:38.431435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b36119-1bf9-4653-b746-aeb0214f142f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:38.431435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b36119-1bf9-4653-b746-aeb0214f142f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:38.503418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b36119-1bf9-4653-b746-aeb0214f142f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:38.504916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b36119-1bf9-4653-b746-aeb0214f142f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:38.612790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 572/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:21:38.619582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6de304-3563-43f6-b711-7625333001ec None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:38.647624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:38.728810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6de304-3563-43f6-b711-7625333001ec None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:38.765768 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6acf8-391d-48b9-98cc-09544ed2eeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:38.766698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6acf8-391d-48b9-98cc-09544ed2eeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:38.767272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6acf8-391d-48b9-98cc-09544ed2eeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:38.767545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6acf8-391d-48b9-98cc-09544ed2eeb4 None admin] PATH_INFO: `/v3/projects/aa8ced7a73f14cbb9cb4bd43f2d69848` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:38.768799 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6acf8-391d-48b9-98cc-09544ed2eeb4 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8ced7a73f14cbb9cb4bd43f2d69848)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:38.770257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6acf8-391d-48b9-98cc-09544ed2eeb4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:38.772159 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 573/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:38 2022] GET /identity/v3/projects/aa8ced7a73f14cbb9cb4bd43f2d69848 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:38.778732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b36119-1bf9-4653-b746-aeb0214f142f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:38.821498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:38.903240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b36119-1bf9-4653-b746-aeb0214f142f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:38.941514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedbb19-a9d5-4d37-9100-fe0f080c5e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:38.942282 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedbb19-a9d5-4d37-9100-fe0f080c5e04 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:38.942567 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedbb19-a9d5-4d37-9100-fe0f080c5e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:38.942915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedbb19-a9d5-4d37-9100-fe0f080c5e04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:38.944535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedbb19-a9d5-4d37-9100-fe0f080c5e04 None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=aa8ced7a73f14cbb9cb4bd43f2d69848, user.id=eedc60e6e4fd47938e4bc6280425d17c) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:38.944865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedbb19-a9d5-4d37-9100-fe0f080c5e04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:38.946422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedbb19-a9d5-4d37-9100-fe0f080c5e04 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:38.953691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 573/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:38 2022] GET /identity/v3/role_assignments?user.id=eedc60e6e4fd47938e4bc6280425d17c&scope.project.id=aa8ced7a73f14cbb9cb4bd43f2d69848&role.id=f957e09d047b47cfa0891d25b4181263 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:41.040134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7313c8e-ec86-41b6-9ac7-8baa009aa3ee None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:41.040698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7313c8e-ec86-41b6-9ac7-8baa009aa3ee None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:41.040698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7313c8e-ec86-41b6-9ac7-8baa009aa3ee None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:41.042077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 574/1147] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:41.047411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1414221-48f8-4b1e-910c-0a5079541d7e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:41.047845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1414221-48f8-4b1e-910c-0a5079541d7e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:41.047977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1414221-48f8-4b1e-910c-0a5079541d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:41.064744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a1414221-48f8-4b1e-910c-0a5079541d7e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:41.065944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a1414221-48f8-4b1e-910c-0a5079541d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:41.133205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 574/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:41.156668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7313c8e-ec86-41b6-9ac7-8baa009aa3ee None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:41.250743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:41.261827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7313c8e-ec86-41b6-9ac7-8baa009aa3ee None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:41.305670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db8b9f-30fd-4bec-b129-b55074475486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:41.306398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db8b9f-30fd-4bec-b129-b55074475486 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:41.306683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db8b9f-30fd-4bec-b129-b55074475486 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:41.306966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db8b9f-30fd-4bec-b129-b55074475486 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:41.319591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db8b9f-30fd-4bec-b129-b55074475486 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-e7db8b9f-30fd-4bec-b129-b55074475486 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:41.327468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:41.328756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:41.329652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 575/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:41.335655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1414221-48f8-4b1e-910c-0a5079541d7e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:41.362317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:41.446834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1414221-48f8-4b1e-910c-0a5079541d7e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:41.482822 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b57c7-ff70-42fa-9751-781c5f66bd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:41.483501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b57c7-ff70-42fa-9751-781c5f66bd34 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:41.483782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b57c7-ff70-42fa-9751-781c5f66bd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:41.484055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b57c7-ff70-42fa-9751-781c5f66bd34 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:41.484861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b57c7-ff70-42fa-9751-781c5f66bd34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:41.485166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b57c7-ff70-42fa-9751-781c5f66bd34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:41.486570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b57c7-ff70-42fa-9751-781c5f66bd34 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:41.496271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 575/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:41.502397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db8b9f-30fd-4bec-b129-b55074475486 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:41.642710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db8b9f-30fd-4bec-b129-b55074475486 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:41.675762 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c328c9-314d-4b62-8832-8b87d15c3b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:41.676660 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c328c9-314d-4b62-8832-8b87d15c3b49 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:41.676941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c328c9-314d-4b62-8832-8b87d15c3b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:41.677212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c328c9-314d-4b62-8832-8b87d15c3b49 None admin] PATH_INFO: `/v3/users/eedc60e6e4fd47938e4bc6280425d17c` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:41.679245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c328c9-314d-4b62-8832-8b87d15c3b49 None admin] RBAC: Authorizing `identity:get_user(user_id=eedc60e6e4fd47938e4bc6280425d17c)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:41.680704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c328c9-314d-4b62-8832-8b87d15c3b49 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:41.682327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 576/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:41 2022] GET /identity/v3/users/eedc60e6e4fd47938e4bc6280425d17c => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:41.688202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b57c7-ff70-42fa-9751-781c5f66bd34 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:41.778230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:41.795803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b57c7-ff70-42fa-9751-781c5f66bd34 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:41.829856 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47aca7-5b6a-4a82-9bec-31ea5a3dc4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:41.830713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47aca7-5b6a-4a82-9bec-31ea5a3dc4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:41.830994 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47aca7-5b6a-4a82-9bec-31ea5a3dc4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:41.831267 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47aca7-5b6a-4a82-9bec-31ea5a3dc4d6 None admin] PATH_INFO: `/v3/projects/aa8ced7a73f14cbb9cb4bd43f2d69848` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:41.832431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47aca7-5b6a-4a82-9bec-31ea5a3dc4d6 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8ced7a73f14cbb9cb4bd43f2d69848)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:41.833919 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47aca7-5b6a-4a82-9bec-31ea5a3dc4d6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:41.835620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 576/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:41 2022] GET /identity/v3/projects/aa8ced7a73f14cbb9cb4bd43f2d69848 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:41.841578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c328c9-314d-4b62-8832-8b87d15c3b49 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:41.864654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:41.949979 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c328c9-314d-4b62-8832-8b87d15c3b49 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:41.983755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4e29c-a7ce-44fb-982c-bd715c3f7f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:41.984689 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4e29c-a7ce-44fb-982c-bd715c3f7f7b None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:41.984975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4e29c-a7ce-44fb-982c-bd715c3f7f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:41.985252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4e29c-a7ce-44fb-982c-bd715c3f7f7b None admin] PATH_INFO: `/v3/projects/aa8ced7a73f14cbb9cb4bd43f2d69848/users/eedc60e6e4fd47938e4bc6280425d17c/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:41.987209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4e29c-a7ce-44fb-982c-bd715c3f7f7b None admin] RBAC: Authorizing `identity:create_grant(project_id=aa8ced7a73f14cbb9cb4bd43f2d69848, user_id=eedc60e6e4fd47938e4bc6280425d17c, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:41.988667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4e29c-a7ce-44fb-982c-bd715c3f7f7b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:42.002540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 577/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:21:41 2022] PUT /identity/v3/projects/aa8ced7a73f14cbb9cb4bd43f2d69848/users/eedc60e6e4fd47938e4bc6280425d17c/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:21:44.120784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44f479-71bc-4cdc-ab3b-3d0dac08dcfe None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:44.121322 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44f479-71bc-4cdc-ab3b-3d0dac08dcfe None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:44.121414 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44f479-71bc-4cdc-ab3b-3d0dac08dcfe None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:44.123059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 577/1154] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:44.129750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007ce95-ebd6-4eb5-991f-79c8833478e2 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:44.130199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007ce95-ebd6-4eb5-991f-79c8833478e2 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:44.131158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007ce95-ebd6-4eb5-991f-79c8833478e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:44.154663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-4007ce95-ebd6-4eb5-991f-79c8833478e2 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:44.156535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4007ce95-ebd6-4eb5-991f-79c8833478e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:44.246016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 578/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:44.270529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44f479-71bc-4cdc-ab3b-3d0dac08dcfe None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:44.394233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44f479-71bc-4cdc-ab3b-3d0dac08dcfe None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:44.431956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbb447-3927-445e-bd6c-b240e2aa1618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:44.433177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbb447-3927-445e-bd6c-b240e2aa1618 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:44.433177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbb447-3927-445e-bd6c-b240e2aa1618 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:44.433360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbb447-3927-445e-bd6c-b240e2aa1618 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:44.446305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbb447-3927-445e-bd6c-b240e2aa1618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-21bbb447-3927-445e-bd6c-b240e2aa1618 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:44.454244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:44.455477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:44.456512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 578/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:44.462296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007ce95-ebd6-4eb5-991f-79c8833478e2 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:44.475187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:44.576723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007ce95-ebd6-4eb5-991f-79c8833478e2 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:44.617001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322906a-89aa-4b5b-842c-c5b1ad699eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:44.617989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322906a-89aa-4b5b-842c-c5b1ad699eaf None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:44.618331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322906a-89aa-4b5b-842c-c5b1ad699eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:44.618684 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322906a-89aa-4b5b-842c-c5b1ad699eaf None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:44.619821 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322906a-89aa-4b5b-842c-c5b1ad699eaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:44.620306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322906a-89aa-4b5b-842c-c5b1ad699eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:44.622268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322906a-89aa-4b5b-842c-c5b1ad699eaf None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:44.634155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 579/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:44.640280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbb447-3927-445e-bd6c-b240e2aa1618 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:44.659209 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:44.751563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbb447-3927-445e-bd6c-b240e2aa1618 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:44.788183 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7de01-6629-4832-a7ca-3ac37fb25837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:44.789092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7de01-6629-4832-a7ca-3ac37fb25837 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:44.789368 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7de01-6629-4832-a7ca-3ac37fb25837 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:44.789637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7de01-6629-4832-a7ca-3ac37fb25837 None admin] PATH_INFO: `/v3/users/eedc60e6e4fd47938e4bc6280425d17c` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:44.790828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7de01-6629-4832-a7ca-3ac37fb25837 None admin] RBAC: Authorizing `identity:get_user(user_id=eedc60e6e4fd47938e4bc6280425d17c)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:44.792260 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7de01-6629-4832-a7ca-3ac37fb25837 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:44.793923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 579/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:44 2022] GET /identity/v3/users/eedc60e6e4fd47938e4bc6280425d17c => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:44.799772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322906a-89aa-4b5b-842c-c5b1ad699eaf None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:44.868511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:44.897688 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322906a-89aa-4b5b-842c-c5b1ad699eaf None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:44.930534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6840dd6-25a6-4586-a43a-cdec6159838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:44.931110 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6840dd6-25a6-4586-a43a-cdec6159838a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:44.931387 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6840dd6-25a6-4586-a43a-cdec6159838a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:44.931662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6840dd6-25a6-4586-a43a-cdec6159838a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:44.997127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6840dd6-25a6-4586-a43a-cdec6159838a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:44.998566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6840dd6-25a6-4586-a43a-cdec6159838a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:45.064305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:45.094793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 580/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:21:45.101091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7de01-6629-4832-a7ca-3ac37fb25837 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:45.181670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:45.200208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7de01-6629-4832-a7ca-3ac37fb25837 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:45.233677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a45eb-03f4-44a2-b525-d593c5c461d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:45.234536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a45eb-03f4-44a2-b525-d593c5c461d5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:45.234822 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a45eb-03f4-44a2-b525-d593c5c461d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:45.235101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a45eb-03f4-44a2-b525-d593c5c461d5 None admin] PATH_INFO: `/v3/projects/aa8ced7a73f14cbb9cb4bd43f2d69848` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:45.236288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a45eb-03f4-44a2-b525-d593c5c461d5 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8ced7a73f14cbb9cb4bd43f2d69848)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:45.237805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a45eb-03f4-44a2-b525-d593c5c461d5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:45.239661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 580/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:45 2022] GET /identity/v3/projects/aa8ced7a73f14cbb9cb4bd43f2d69848 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:21:45.246344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6840dd6-25a6-4586-a43a-cdec6159838a None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:45.342324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6840dd6-25a6-4586-a43a-cdec6159838a None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:45.377520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77ff2a-8d65-4e8c-a42e-617b14704df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:45.378185 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77ff2a-8d65-4e8c-a42e-617b14704df4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:45.378472 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77ff2a-8d65-4e8c-a42e-617b14704df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:45.378748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77ff2a-8d65-4e8c-a42e-617b14704df4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:45.379590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77ff2a-8d65-4e8c-a42e-617b14704df4 None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=aa8ced7a73f14cbb9cb4bd43f2d69848, user.id=eedc60e6e4fd47938e4bc6280425d17c) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:45.379902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77ff2a-8d65-4e8c-a42e-617b14704df4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:45.382014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77ff2a-8d65-4e8c-a42e-617b14704df4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:45.388110 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 581/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:45 2022] GET /identity/v3/role_assignments?user.id=eedc60e6e4fd47938e4bc6280425d17c&scope.project.id=aa8ced7a73f14cbb9cb4bd43f2d69848&role.id=f957e09d047b47cfa0891d25b4181263 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:47.447390 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647db2a-c0e0-454b-80eb-2d25b917ddfd None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:47.447834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647db2a-c0e0-454b-80eb-2d25b917ddfd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:47.447946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647db2a-c0e0-454b-80eb-2d25b917ddfd None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:47.449354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 581/1162] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:47.454541 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4844d-145f-4015-ac2a-31c75d528470 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:47.454947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4844d-145f-4015-ac2a-31c75d528470 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:47.455131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4844d-145f-4015-ac2a-31c75d528470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:47.472361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-71d4844d-145f-4015-ac2a-31c75d528470 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:47.473589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-71d4844d-145f-4015-ac2a-31c75d528470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:47.502620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:47.540936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 582/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:47.563133 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647db2a-c0e0-454b-80eb-2d25b917ddfd None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:47.666051 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647db2a-c0e0-454b-80eb-2d25b917ddfd None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:47.699245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86872c1f-1db5-4d94-933a-cd09af4736e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:47.699815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86872c1f-1db5-4d94-933a-cd09af4736e8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:47.700101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86872c1f-1db5-4d94-933a-cd09af4736e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:47.700379 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86872c1f-1db5-4d94-933a-cd09af4736e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:47.718849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:47.767551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86872c1f-1db5-4d94-933a-cd09af4736e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:47.769038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86872c1f-1db5-4d94-933a-cd09af4736e8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:47.873736 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 582/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:21:47.880016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4844d-145f-4015-ac2a-31c75d528470 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:47.985913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4844d-145f-4015-ac2a-31c75d528470 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:48.020870 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15ae2e-b70c-44fc-a780-d4fd300afaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:48.021899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15ae2e-b70c-44fc-a780-d4fd300afaab None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:48.022183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15ae2e-b70c-44fc-a780-d4fd300afaab None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:48.022460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15ae2e-b70c-44fc-a780-d4fd300afaab None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:48.029675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15ae2e-b70c-44fc-a780-d4fd300afaab None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:48.031133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15ae2e-b70c-44fc-a780-d4fd300afaab None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-0b15ae2e-b70c-44fc-a780-d4fd300afaab None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:48.046631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:48.047988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:21:48.049103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 583/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 22:21:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:48.055367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86872c1f-1db5-4d94-933a-cd09af4736e8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:48.105289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:48.154447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86872c1f-1db5-4d94-933a-cd09af4736e8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:48.188805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff985370-a662-484a-b462-73527ec13b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:48.189458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff985370-a662-484a-b462-73527ec13b4d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:48.189737 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff985370-a662-484a-b462-73527ec13b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:48.190010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff985370-a662-484a-b462-73527ec13b4d None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:48.190721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff985370-a662-484a-b462-73527ec13b4d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:48.191025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff985370-a662-484a-b462-73527ec13b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:48.192376 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff985370-a662-484a-b462-73527ec13b4d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:48.199368 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 583/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Thu Jul 21 22:21:48 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:50.244447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa4dcf-0011-4e0c-8628-2588c427e97b None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:50.245125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa4dcf-0011-4e0c-8628-2588c427e97b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:50.245125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa4dcf-0011-4e0c-8628-2588c427e97b None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:50.246617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 584/1167] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:50.253050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e21e-5ba8-4a1f-89fc-a2ac666b66ed None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:50.253631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e21e-5ba8-4a1f-89fc-a2ac666b66ed None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:50.253631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3e21e-5ba8-4a1f-89fc-a2ac666b66ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:50.270691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-afc3e21e-5ba8-4a1f-89fc-a2ac666b66ed None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:50.271868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-afc3e21e-5ba8-4a1f-89fc-a2ac666b66ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:50.336546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 584/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:21:50.360727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa4dcf-0011-4e0c-8628-2588c427e97b None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:50.391969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:50.467895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa4dcf-0011-4e0c-8628-2588c427e97b None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:50.501451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8aac0-d9f5-48ff-b8e0-e6860fed62e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:50.502035 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8aac0-d9f5-48ff-b8e0-e6860fed62e3 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:50.502316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8aac0-d9f5-48ff-b8e0-e6860fed62e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:50.502590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8aac0-d9f5-48ff-b8e0-e6860fed62e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:50.503332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8aac0-d9f5-48ff-b8e0-e6860fed62e3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:50.504712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8aac0-d9f5-48ff-b8e0-e6860fed62e3 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:50.519058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 585/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Thu Jul 21 22:21:50 2022] POST /identity/v3/domains => generated 267 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:21:52.587732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060352d4-8588-4406-ade6-2c6790ed65e6 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:52.588280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060352d4-8588-4406-ade6-2c6790ed65e6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:52.588280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060352d4-8588-4406-ade6-2c6790ed65e6 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:52.589724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 585/1170] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:52.594899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c02c3-29fb-4dea-94b1-67ded47f2c6d None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:52.595339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c02c3-29fb-4dea-94b1-67ded47f2c6d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:52.595447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c02c3-29fb-4dea-94b1-67ded47f2c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:52.612289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7b0c02c3-29fb-4dea-94b1-67ded47f2c6d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:52.613498 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7b0c02c3-29fb-4dea-94b1-67ded47f2c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:52.679483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 586/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:21:52.702931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060352d4-8588-4406-ade6-2c6790ed65e6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:52.749206 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:52.806756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060352d4-8588-4406-ade6-2c6790ed65e6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:52.841356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b233f86-6cae-4ab6-bdfe-cfcb0ff8e404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:52.842211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b233f86-6cae-4ab6-bdfe-cfcb0ff8e404 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:52.842493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b233f86-6cae-4ab6-bdfe-cfcb0ff8e404 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:52.842765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b233f86-6cae-4ab6-bdfe-cfcb0ff8e404 None admin] PATH_INFO: `/v3/domains/07f9ad074ca9450db353c28c1320f5bf` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:52.854209 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b233f86-6cae-4ab6-bdfe-cfcb0ff8e404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=07f9ad074ca9450db353c28c1320f5bf)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:52.855661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b233f86-6cae-4ab6-bdfe-cfcb0ff8e404 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:52.857348 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 586/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Thu Jul 21 22:21:52 2022] GET /identity/v3/domains/07f9ad074ca9450db353c28c1320f5bf => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:52.863472 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c02c3-29fb-4dea-94b1-67ded47f2c6d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:52.880876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:52.962467 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c02c3-29fb-4dea-94b1-67ded47f2c6d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:52.996454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a807d9-efd2-4945-8288-334f79dc9635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:52.997140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a807d9-efd2-4945-8288-334f79dc9635 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:52.997425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a807d9-efd2-4945-8288-334f79dc9635 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:52.997703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a807d9-efd2-4945-8288-334f79dc9635 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:52.998420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a807d9-efd2-4945-8288-334f79dc9635 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:52.999843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a807d9-efd2-4945-8288-334f79dc9635 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:53.025866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 587/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:21:52 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:21:55.123279 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328ce93-3c39-4b4d-b479-5ddc47b3d91c None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:55.123842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328ce93-3c39-4b4d-b479-5ddc47b3d91c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:55.123842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328ce93-3c39-4b4d-b479-5ddc47b3d91c None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:55.125256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 587/1174] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:55.130561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedc4d3-ec16-4d35-802e-53e5c06fe34c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:55.130973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedc4d3-ec16-4d35-802e-53e5c06fe34c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:55.131126 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedc4d3-ec16-4d35-802e-53e5c06fe34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:55.149149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-2bedc4d3-ec16-4d35-802e-53e5c06fe34c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:55.150333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2bedc4d3-ec16-4d35-802e-53e5c06fe34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:55.211489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:55.217723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 588/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:55.242932 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328ce93-3c39-4b4d-b479-5ddc47b3d91c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:55.343634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328ce93-3c39-4b4d-b479-5ddc47b3d91c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:55.345940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:55.376663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d7e82-5190-4b51-9604-79d2cce3d94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:55.377444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d7e82-5190-4b51-9604-79d2cce3d94f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:55.377731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d7e82-5190-4b51-9604-79d2cce3d94f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:55.378009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d7e82-5190-4b51-9604-79d2cce3d94f None admin] PATH_INFO: `/v3/domains/07f9ad074ca9450db353c28c1320f5bf` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:55.379203 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d7e82-5190-4b51-9604-79d2cce3d94f None admin] RBAC: Authorizing `identity:get_domain(domain_id=07f9ad074ca9450db353c28c1320f5bf)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:55.381332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d7e82-5190-4b51-9604-79d2cce3d94f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:55.382977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 588/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Thu Jul 21 22:21:55 2022] GET /identity/v3/domains/07f9ad074ca9450db353c28c1320f5bf => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:55.388882 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedc4d3-ec16-4d35-802e-53e5c06fe34c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:55.487199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedc4d3-ec16-4d35-802e-53e5c06fe34c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:55.520344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a020c5-77a1-4540-9fc9-5763f032e4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:55.521057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a020c5-77a1-4540-9fc9-5763f032e4aa None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:55.521331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a020c5-77a1-4540-9fc9-5763f032e4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:55.521613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a020c5-77a1-4540-9fc9-5763f032e4aa None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:55.522327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a020c5-77a1-4540-9fc9-5763f032e4aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:55.523760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a020c5-77a1-4540-9fc9-5763f032e4aa None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:55.543023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 589/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:21:55 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:21:57.696531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a918181-63bb-480b-b92f-d1b539835b16 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:57.697050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a918181-63bb-480b-b92f-d1b539835b16 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:57.697050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a918181-63bb-480b-b92f-d1b539835b16 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:57.698383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 589/1178] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:21:57.703726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e069-895d-4405-b8ba-da84215de329 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:57.704176 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e069-895d-4405-b8ba-da84215de329 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:57.704290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e069-895d-4405-b8ba-da84215de329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:57.720951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-9bc9e069-895d-4405-b8ba-da84215de329 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:21:57.722145 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9bc9e069-895d-4405-b8ba-da84215de329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:21:57.771118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:57.789324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 590/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:21:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:21:57.813076 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a918181-63bb-480b-b92f-d1b539835b16 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:57.914175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a918181-63bb-480b-b92f-d1b539835b16 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:57.947054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ca65b-d2fa-428b-8819-73f510b17766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:57.947773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ca65b-d2fa-428b-8819-73f510b17766 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:57.948060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ca65b-d2fa-428b-8819-73f510b17766 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:57.948341 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ca65b-d2fa-428b-8819-73f510b17766 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:57.959367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ca65b-d2fa-428b-8819-73f510b17766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-c15ca65b-d2fa-428b-8819-73f510b17766 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:21:57.967052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:21:57.968385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:21:57.969727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 590/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:21:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:21:57.974994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e069-895d-4405-b8ba-da84215de329 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:58.075276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e069-895d-4405-b8ba-da84215de329 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:58.108922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe663c-a70e-4c89-9963-78d91ac6d0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:58.109604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe663c-a70e-4c89-9963-78d91ac6d0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:58.109892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe663c-a70e-4c89-9963-78d91ac6d0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:58.110174 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe663c-a70e-4c89-9963-78d91ac6d0f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:58.110927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe663c-a70e-4c89-9963-78d91ac6d0f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:58.111268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe663c-a70e-4c89-9963-78d91ac6d0f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:58.112808 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe663c-a70e-4c89-9963-78d91ac6d0f6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:58.122313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 591/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:21:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:58.128170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ca65b-d2fa-428b-8819-73f510b17766 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:58.159482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:58.226555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ca65b-d2fa-428b-8819-73f510b17766 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:58.260121 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599c7be-b0cb-432f-b89a-cd86b7e84908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:58.261015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599c7be-b0cb-432f-b89a-cd86b7e84908 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:58.261301 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599c7be-b0cb-432f-b89a-cd86b7e84908 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:58.261579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599c7be-b0cb-432f-b89a-cd86b7e84908 None admin] PATH_INFO: `/v3/users/05dd5ece4ba040d2914f4861157f994a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:58.275651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599c7be-b0cb-432f-b89a-cd86b7e84908 None admin] RBAC: Authorizing `identity:get_user(user_id=05dd5ece4ba040d2914f4861157f994a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:58.277135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599c7be-b0cb-432f-b89a-cd86b7e84908 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:58.278799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 591/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:21:58 2022] GET /identity/v3/users/05dd5ece4ba040d2914f4861157f994a => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:58.284686 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe663c-a70e-4c89-9963-78d91ac6d0f6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:58.330464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:58.382916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe663c-a70e-4c89-9963-78d91ac6d0f6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:58.415852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b514c-706b-4d5e-bb47-65cd9ad316c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:58.416529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b514c-706b-4d5e-bb47-65cd9ad316c2 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:58.416824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b514c-706b-4d5e-bb47-65cd9ad316c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:58.417109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b514c-706b-4d5e-bb47-65cd9ad316c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:58.485349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b514c-706b-4d5e-bb47-65cd9ad316c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:58.486727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b514c-706b-4d5e-bb47-65cd9ad316c2 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:58.533045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:58.587652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 592/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:21:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:21:58.593573 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599c7be-b0cb-432f-b89a-cd86b7e84908 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:58.682700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:21:58.693435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599c7be-b0cb-432f-b89a-cd86b7e84908 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:58.728118 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5ef63-d23e-46b2-885a-501219829e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:58.729008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5ef63-d23e-46b2-885a-501219829e94 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:58.729292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5ef63-d23e-46b2-885a-501219829e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:58.729568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5ef63-d23e-46b2-885a-501219829e94 None admin] PATH_INFO: `/v3/projects/43939e56233e46f0a329dce8a8454cc1` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:58.730742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5ef63-d23e-46b2-885a-501219829e94 None admin] RBAC: Authorizing `identity:get_project(project_id=43939e56233e46f0a329dce8a8454cc1)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:58.732184 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5ef63-d23e-46b2-885a-501219829e94 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:58.733923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 592/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:21:58 2022] GET /identity/v3/projects/43939e56233e46f0a329dce8a8454cc1 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:21:58.740761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b514c-706b-4d5e-bb47-65cd9ad316c2 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:21:58.838937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b514c-706b-4d5e-bb47-65cd9ad316c2 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:21:58.872500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e2411-ecb7-4b57-9e84-5e9f6c35fd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:21:58.873171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e2411-ecb7-4b57-9e84-5e9f6c35fd4d None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:21:58.873455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e2411-ecb7-4b57-9e84-5e9f6c35fd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:21:58.873731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e2411-ecb7-4b57-9e84-5e9f6c35fd4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:21:58.874562 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e2411-ecb7-4b57-9e84-5e9f6c35fd4d None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=43939e56233e46f0a329dce8a8454cc1, user.id=05dd5ece4ba040d2914f4861157f994a) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:21:58.874874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e2411-ecb7-4b57-9e84-5e9f6c35fd4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:21:58.876299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e2411-ecb7-4b57-9e84-5e9f6c35fd4d None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:21:58.882434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 593/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:21:58 2022] GET /identity/v3/role_assignments?user.id=05dd5ece4ba040d2914f4861157f994a&scope.project.id=43939e56233e46f0a329dce8a8454cc1&role.id=f957e09d047b47cfa0891d25b4181263 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:00.915490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560b7fa-a33d-49e8-be44-a96534b5e43e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:00.916035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560b7fa-a33d-49e8-be44-a96534b5e43e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:00.916035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560b7fa-a33d-49e8-be44-a96534b5e43e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:00.917468 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 593/1186] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:22:00.923316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845078d9-c4d0-49f6-a9dc-24c3bf100f92 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:00.923816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845078d9-c4d0-49f6-a9dc-24c3bf100f92 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:00.923906 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845078d9-c4d0-49f6-a9dc-24c3bf100f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:00.942452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-845078d9-c4d0-49f6-a9dc-24c3bf100f92 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:22:00.943672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-845078d9-c4d0-49f6-a9dc-24c3bf100f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:22:00.946982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:01.014581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 594/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:22:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:22:01.037807 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560b7fa-a33d-49e8-be44-a96534b5e43e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:01.155337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560b7fa-a33d-49e8-be44-a96534b5e43e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:01.194764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae58ace-a2c2-46c6-9718-e9e4ea1d4184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:01.195486 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae58ace-a2c2-46c6-9718-e9e4ea1d4184 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:01.195766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae58ace-a2c2-46c6-9718-e9e4ea1d4184 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:01.196038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae58ace-a2c2-46c6-9718-e9e4ea1d4184 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:01.206273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae58ace-a2c2-46c6-9718-e9e4ea1d4184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-cae58ace-a2c2-46c6-9718-e9e4ea1d4184 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:22:01.213895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:01.215025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:22:01.216332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 594/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:22:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:01.221194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845078d9-c4d0-49f6-a9dc-24c3bf100f92 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:01.317987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845078d9-c4d0-49f6-a9dc-24c3bf100f92 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:01.351780 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c037e-0670-44c5-b768-2434201b6adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:01.352441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c037e-0670-44c5-b768-2434201b6adc None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:01.352753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c037e-0670-44c5-b768-2434201b6adc None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:01.353025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c037e-0670-44c5-b768-2434201b6adc None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:01.353769 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c037e-0670-44c5-b768-2434201b6adc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:01.354075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c037e-0670-44c5-b768-2434201b6adc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:01.355418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c037e-0670-44c5-b768-2434201b6adc None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:01.363431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 595/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:22:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:01.370624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae58ace-a2c2-46c6-9718-e9e4ea1d4184 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:01.390390 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:01.487527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae58ace-a2c2-46c6-9718-e9e4ea1d4184 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:01.522812 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318afaa-74d3-42e4-ab62-dae1805f34b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:01.523685 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318afaa-74d3-42e4-ab62-dae1805f34b5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:01.523969 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318afaa-74d3-42e4-ab62-dae1805f34b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:01.524246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318afaa-74d3-42e4-ab62-dae1805f34b5 None admin] PATH_INFO: `/v3/users/05dd5ece4ba040d2914f4861157f994a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:01.525480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318afaa-74d3-42e4-ab62-dae1805f34b5 None admin] RBAC: Authorizing `identity:get_user(user_id=05dd5ece4ba040d2914f4861157f994a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:01.526926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318afaa-74d3-42e4-ab62-dae1805f34b5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:01.528599 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 595/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:22:01 2022] GET /identity/v3/users/05dd5ece4ba040d2914f4861157f994a => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:01.534539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c037e-0670-44c5-b768-2434201b6adc None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:01.539447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:01.636789 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c037e-0670-44c5-b768-2434201b6adc None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:01.670983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8baf2a-ab7b-4a34-b505-e41506b2d0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:01.671852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8baf2a-ab7b-4a34-b505-e41506b2d0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:01.672164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8baf2a-ab7b-4a34-b505-e41506b2d0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:01.672442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8baf2a-ab7b-4a34-b505-e41506b2d0b4 None admin] PATH_INFO: `/v3/projects/43939e56233e46f0a329dce8a8454cc1` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:01.673656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8baf2a-ab7b-4a34-b505-e41506b2d0b4 None admin] RBAC: Authorizing `identity:get_project(project_id=43939e56233e46f0a329dce8a8454cc1)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:01.675083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8baf2a-ab7b-4a34-b505-e41506b2d0b4 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:01.677999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 596/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:22:01 2022] GET /identity/v3/projects/43939e56233e46f0a329dce8a8454cc1 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:01.684068 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318afaa-74d3-42e4-ab62-dae1805f34b5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:01.770810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:01.786626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318afaa-74d3-42e4-ab62-dae1805f34b5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:01.829454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d138284-a4ca-4f2c-b827-114caa2e9580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:01.830863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d138284-a4ca-4f2c-b827-114caa2e9580 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:01.831347 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d138284-a4ca-4f2c-b827-114caa2e9580 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:01.831814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d138284-a4ca-4f2c-b827-114caa2e9580 None admin] PATH_INFO: `/v3/projects/43939e56233e46f0a329dce8a8454cc1/users/05dd5ece4ba040d2914f4861157f994a/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:01.835412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d138284-a4ca-4f2c-b827-114caa2e9580 None admin] RBAC: Authorizing `identity:create_grant(project_id=43939e56233e46f0a329dce8a8454cc1, user_id=05dd5ece4ba040d2914f4861157f994a, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:01.837899 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d138284-a4ca-4f2c-b827-114caa2e9580 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:01.854507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 596/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:22:01 2022] PUT /identity/v3/projects/43939e56233e46f0a329dce8a8454cc1/users/05dd5ece4ba040d2914f4861157f994a/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:22:03.886904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76913f-c674-419e-b7df-84e68b42a0a6 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:03.887471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76913f-c674-419e-b7df-84e68b42a0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:03.887471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76913f-c674-419e-b7df-84e68b42a0a6 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:03.888875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 597/1193] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:22:03.894238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0feb99-a898-4a53-a8ef-a08e9f4b598a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:03.894716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0feb99-a898-4a53-a8ef-a08e9f4b598a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:03.895429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0feb99-a898-4a53-a8ef-a08e9f4b598a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:03.912875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ce0feb99-a898-4a53-a8ef-a08e9f4b598a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:22:03.914063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ce0feb99-a898-4a53-a8ef-a08e9f4b598a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:22:03.980563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 597/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:22:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:22:04.003978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76913f-c674-419e-b7df-84e68b42a0a6 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:04.074543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:04.105847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76913f-c674-419e-b7df-84e68b42a0a6 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:04.143144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6ba69-04db-46b1-a02f-2b2b9e998af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:04.143886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6ba69-04db-46b1-a02f-2b2b9e998af7 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:04.144173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6ba69-04db-46b1-a02f-2b2b9e998af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:04.144450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6ba69-04db-46b1-a02f-2b2b9e998af7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:04.156854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6ba69-04db-46b1-a02f-2b2b9e998af7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-fca6ba69-04db-46b1-a02f-2b2b9e998af7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:22:04.165151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:22:04.166427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:22:04.167500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 598/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:22:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:22:04.172645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0feb99-a898-4a53-a8ef-a08e9f4b598a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:04.248061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:04.273911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0feb99-a898-4a53-a8ef-a08e9f4b598a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:04.309841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce281fc-1eaa-4afb-a7cb-9a5ac3581b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:04.310508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce281fc-1eaa-4afb-a7cb-9a5ac3581b8f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:04.310790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce281fc-1eaa-4afb-a7cb-9a5ac3581b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:04.311066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce281fc-1eaa-4afb-a7cb-9a5ac3581b8f None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:04.311817 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce281fc-1eaa-4afb-a7cb-9a5ac3581b8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:04.312124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce281fc-1eaa-4afb-a7cb-9a5ac3581b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:04.313544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce281fc-1eaa-4afb-a7cb-9a5ac3581b8f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:04.322438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 598/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:22:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:04.328223 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6ba69-04db-46b1-a02f-2b2b9e998af7 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:04.427255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6ba69-04db-46b1-a02f-2b2b9e998af7 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:04.454958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:04.461139 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1678ab-af44-43cf-b789-f93c048811f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:04.462018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1678ab-af44-43cf-b789-f93c048811f1 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:04.462305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1678ab-af44-43cf-b789-f93c048811f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:04.462582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1678ab-af44-43cf-b789-f93c048811f1 None admin] PATH_INFO: `/v3/users/05dd5ece4ba040d2914f4861157f994a` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:04.463776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1678ab-af44-43cf-b789-f93c048811f1 None admin] RBAC: Authorizing `identity:get_user(user_id=05dd5ece4ba040d2914f4861157f994a)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:04.465278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1678ab-af44-43cf-b789-f93c048811f1 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:04.466902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 599/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Thu Jul 21 22:22:04 2022] GET /identity/v3/users/05dd5ece4ba040d2914f4861157f994a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:04.473277 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce281fc-1eaa-4afb-a7cb-9a5ac3581b8f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:04.571764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce281fc-1eaa-4afb-a7cb-9a5ac3581b8f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:04.605509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223468-ee46-41e9-8803-f314d724bc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:04.606157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223468-ee46-41e9-8803-f314d724bc6a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:04.606443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223468-ee46-41e9-8803-f314d724bc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:04.606720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223468-ee46-41e9-8803-f314d724bc6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:04.615486 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:04.674787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10223468-ee46-41e9-8803-f314d724bc6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:04.676804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10223468-ee46-41e9-8803-f314d724bc6a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:04.775406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 599/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:22:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:22:04.781480 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1678ab-af44-43cf-b789-f93c048811f1 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:04.887138 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1678ab-af44-43cf-b789-f93c048811f1 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:04.924197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee00747-ef47-44f2-97f3-9e1f3c67d004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:04.925115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee00747-ef47-44f2-97f3-9e1f3c67d004 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:04.925400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee00747-ef47-44f2-97f3-9e1f3c67d004 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:04.925675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee00747-ef47-44f2-97f3-9e1f3c67d004 None admin] PATH_INFO: `/v3/projects/43939e56233e46f0a329dce8a8454cc1` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:04.927030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee00747-ef47-44f2-97f3-9e1f3c67d004 None admin] RBAC: Authorizing `identity:get_project(project_id=43939e56233e46f0a329dce8a8454cc1)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:04.928509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee00747-ef47-44f2-97f3-9e1f3c67d004 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:04.930320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 600/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Thu Jul 21 22:22:04 2022] GET /identity/v3/projects/43939e56233e46f0a329dce8a8454cc1 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:04.936756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223468-ee46-41e9-8803-f314d724bc6a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:22:04.977954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:05.040988 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223468-ee46-41e9-8803-f314d724bc6a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:22:05.075593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d0cd2-87c0-4a85-92ac-159b87d86037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:22:05.076250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d0cd2-87c0-4a85-92ac-159b87d86037 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:22:05.076564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d0cd2-87c0-4a85-92ac-159b87d86037 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:22:05.076843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d0cd2-87c0-4a85-92ac-159b87d86037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:22:05.077675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d0cd2-87c0-4a85-92ac-159b87d86037 None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=43939e56233e46f0a329dce8a8454cc1, user.id=05dd5ece4ba040d2914f4861157f994a) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:22:05.077991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d0cd2-87c0-4a85-92ac-159b87d86037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:22:05.079432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d0cd2-87c0-4a85-92ac-159b87d86037 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:22:05.086642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 600/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:22:04 2022] GET /identity/v3/role_assignments?user.id=05dd5ece4ba040d2914f4861157f994a&scope.project.id=43939e56233e46f0a329dce8a8454cc1&role.id=f957e09d047b47cfa0891d25b4181263 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:22:14.919767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=67 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:22:15.083911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=49 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:04.154445 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e526c-f232-41f5-a765-8acd7c3d8071 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:04.155039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e526c-f232-41f5-a765-8acd7c3d8071 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:04.155039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e526c-f232-41f5-a765-8acd7c3d8071 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:04.156429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 601/1201] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:04.162228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584776a8-a9ff-4bd7-8419-a264c169c9e9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:04.162810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584776a8-a9ff-4bd7-8419-a264c169c9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:04.162810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584776a8-a9ff-4bd7-8419-a264c169c9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:04.182917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-584776a8-a9ff-4bd7-8419-a264c169c9e9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:04.184224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-584776a8-a9ff-4bd7-8419-a264c169c9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:04.282844 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 601/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:04.307300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e526c-f232-41f5-a765-8acd7c3d8071 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:04.432055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e526c-f232-41f5-a765-8acd7c3d8071 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:04.478140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1758f-404d-459e-bba5-9b3e36a8952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:04.478981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1758f-404d-459e-bba5-9b3e36a8952f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:04.479296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1758f-404d-459e-bba5-9b3e36a8952f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:04.479555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1758f-404d-459e-bba5-9b3e36a8952f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:04.491931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1758f-404d-459e-bba5-9b3e36a8952f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:04.493460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f1758f-404d-459e-bba5-9b3e36a8952f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-d2f1758f-404d-459e-bba5-9b3e36a8952f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:04.515183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:04.517011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:04.518004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:04.518004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:04.518004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:04.518004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:04.522702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 602/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:23:04.524042 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584776a8-a9ff-4bd7-8419-a264c169c9e9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:04.677184 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584776a8-a9ff-4bd7-8419-a264c169c9e9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:04.724749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:04.735077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91362738-3f56-4834-85ff-13ef79889906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:04.736774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91362738-3f56-4834-85ff-13ef79889906 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:04.736774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91362738-3f56-4834-85ff-13ef79889906 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:04.736774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91362738-3f56-4834-85ff-13ef79889906 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:04.737305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91362738-3f56-4834-85ff-13ef79889906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:04.737628 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91362738-3f56-4834-85ff-13ef79889906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:04.739011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91362738-3f56-4834-85ff-13ef79889906 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:04.756005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 602/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:04.762824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1758f-404d-459e-bba5-9b3e36a8952f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:04.831407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:04.898388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1758f-404d-459e-bba5-9b3e36a8952f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:04.936126 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87a974-67b7-4789-8e7f-e79f74268557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:04.937170 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87a974-67b7-4789-8e7f-e79f74268557 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:04.938107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87a974-67b7-4789-8e7f-e79f74268557 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:04.938546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87a974-67b7-4789-8e7f-e79f74268557 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:04.939403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87a974-67b7-4789-8e7f-e79f74268557 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:04.941062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87a974-67b7-4789-8e7f-e79f74268557 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:04.970318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 603/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Thu Jul 21 22:23:04 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:23:07.321949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d00b2-66f0-40be-af67-ef88f482e126 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:07.322495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d00b2-66f0-40be-af67-ef88f482e126 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:07.322758 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d00b2-66f0-40be-af67-ef88f482e126 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:07.324324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 603/1206] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:23:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:23:07.331406 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859fa04-7c8d-4265-8c02-439821ce7a0d None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:07.331841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859fa04-7c8d-4265-8c02-439821ce7a0d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:07.332034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859fa04-7c8d-4265-8c02-439821ce7a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:07.350328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f859fa04-7c8d-4265-8c02-439821ce7a0d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:07.351638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f859fa04-7c8d-4265-8c02-439821ce7a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:07.432527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 604/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:07.460067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d00b2-66f0-40be-af67-ef88f482e126 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:07.589898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d00b2-66f0-40be-af67-ef88f482e126 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:07.630798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eff94-ee48-4580-bae5-6ee311b9b132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:07.632047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eff94-ee48-4580-bae5-6ee311b9b132 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:07.632335 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eff94-ee48-4580-bae5-6ee311b9b132 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:07.632693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eff94-ee48-4580-bae5-6ee311b9b132 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:07.647354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431eff94-ee48-4580-bae5-6ee311b9b132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-431eff94-ee48-4580-bae5-6ee311b9b132 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:23:07.656315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:07.657992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:07.659631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:07.659631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:07.659631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:07.659631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:07.659631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:23:07.659631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:23:07.659631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:23:07.659631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:23:07.659631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:23:07.659631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:23:07.659631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:23:07.659631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:07.663391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 604/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:23:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:07.665395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859fa04-7c8d-4265-8c02-439821ce7a0d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:07.721873 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:07.794602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859fa04-7c8d-4265-8c02-439821ce7a0d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:07.838790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaac6d0-4c8e-4af4-9007-b0f70c2dad12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:07.839478 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaac6d0-4c8e-4af4-9007-b0f70c2dad12 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:07.839769 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaac6d0-4c8e-4af4-9007-b0f70c2dad12 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:07.840099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaac6d0-4c8e-4af4-9007-b0f70c2dad12 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:07.840845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaac6d0-4c8e-4af4-9007-b0f70c2dad12 None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:07.841156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaac6d0-4c8e-4af4-9007-b0f70c2dad12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:07.842711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaac6d0-4c8e-4af4-9007-b0f70c2dad12 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:07.853794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 605/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:23:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:07.860590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eff94-ee48-4580-bae5-6ee311b9b132 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:07.925973 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:07.994639 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eff94-ee48-4580-bae5-6ee311b9b132 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:08.037089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb54efa-86c1-4713-ba76-f3b8ae436065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:08.038060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb54efa-86c1-4713-ba76-f3b8ae436065 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:08.038445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb54efa-86c1-4713-ba76-f3b8ae436065 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:08.038811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb54efa-86c1-4713-ba76-f3b8ae436065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:08.052660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb54efa-86c1-4713-ba76-f3b8ae436065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:08.054495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb54efa-86c1-4713-ba76-f3b8ae436065 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-9fb54efa-86c1-4713-ba76-f3b8ae436065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:08.075552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:08.076879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:08.078216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:08.078216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:08.078216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:08.078216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:08.078216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 605/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:08.085364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaac6d0-4c8e-4af4-9007-b0f70c2dad12 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:08.227285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:08.230697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaac6d0-4c8e-4af4-9007-b0f70c2dad12 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:08.279569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8d50c-167b-46ef-9dd8-04373c6a2aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:08.280289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8d50c-167b-46ef-9dd8-04373c6a2aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:08.281236 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8d50c-167b-46ef-9dd8-04373c6a2aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:08.281527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8d50c-167b-46ef-9dd8-04373c6a2aa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:08.282447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8d50c-167b-46ef-9dd8-04373c6a2aa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:08.282843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8d50c-167b-46ef-9dd8-04373c6a2aa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:08.284527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8d50c-167b-46ef-9dd8-04373c6a2aa5 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:08.299914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 606/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:08.307112 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb54efa-86c1-4713-ba76-f3b8ae436065 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:08.496172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb54efa-86c1-4713-ba76-f3b8ae436065 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:08.498510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:08.551449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae31b3-f80c-4e02-8313-9281dfdea128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:08.552324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae31b3-f80c-4e02-8313-9281dfdea128 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:08.552638 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae31b3-f80c-4e02-8313-9281dfdea128 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:08.553034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae31b3-f80c-4e02-8313-9281dfdea128 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:08.564505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae31b3-f80c-4e02-8313-9281dfdea128 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:08.566317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae31b3-f80c-4e02-8313-9281dfdea128 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-86ae31b3-f80c-4e02-8313-9281dfdea128 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:23:08.586369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:08.587512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:08.588539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:08.588539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:08.588539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:23:08.588539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:23:08.588539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:23:08.588539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:23:08.588539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:23:08.588539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:23:08.588539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:23:08.588539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:08.588539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 606/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:08.595994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8d50c-167b-46ef-9dd8-04373c6a2aa5 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:08.715935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8d50c-167b-46ef-9dd8-04373c6a2aa5 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:08.754186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f906f57-c108-4912-946f-598946c86620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:08.754800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f906f57-c108-4912-946f-598946c86620 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:08.755087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f906f57-c108-4912-946f-598946c86620 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:08.755363 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f906f57-c108-4912-946f-598946c86620 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:08.756225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f906f57-c108-4912-946f-598946c86620 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:08.756859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f906f57-c108-4912-946f-598946c86620 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:08.761396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f906f57-c108-4912-946f-598946c86620 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:08.781954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 607/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jul 21 22:23:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:08.787888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae31b3-f80c-4e02-8313-9281dfdea128 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:08.919034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae31b3-f80c-4e02-8313-9281dfdea128 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:08.972405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4f08f-bfef-4925-bc8e-2ea6949144eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:08.973404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f08f-bfef-4925-bc8e-2ea6949144eb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:08.973838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f08f-bfef-4925-bc8e-2ea6949144eb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:08.974248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4f08f-bfef-4925-bc8e-2ea6949144eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:09.025538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:09.089229 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4f08f-bfef-4925-bc8e-2ea6949144eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:09.090879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4f08f-bfef-4925-bc8e-2ea6949144eb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:09.217124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 607/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:23:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:23:09.224269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f906f57-c108-4912-946f-598946c86620 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:09.246289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:09.363617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f906f57-c108-4912-946f-598946c86620 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:09.421952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27192b6e-4839-4f77-bde5-02adaafa2c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:09.423171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27192b6e-4839-4f77-bde5-02adaafa2c27 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:09.423685 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27192b6e-4839-4f77-bde5-02adaafa2c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:09.423964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27192b6e-4839-4f77-bde5-02adaafa2c27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:09.440890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27192b6e-4839-4f77-bde5-02adaafa2c27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:09.442828 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27192b6e-4839-4f77-bde5-02adaafa2c27 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-27192b6e-4839-4f77-bde5-02adaafa2c27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:09.463532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:09.464615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:09.465965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:09.465965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:09.465965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:09.465965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:09.465965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 608/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:09.470629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4f08f-bfef-4925-bc8e-2ea6949144eb None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:09.537947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:09.583365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4f08f-bfef-4925-bc8e-2ea6949144eb None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:09.621179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eebc6d-5df1-4849-aeb4-cbfe3a96e674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:09.621758 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eebc6d-5df1-4849-aeb4-cbfe3a96e674 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:09.622039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eebc6d-5df1-4849-aeb4-cbfe3a96e674 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:09.622315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eebc6d-5df1-4849-aeb4-cbfe3a96e674 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:09.623050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eebc6d-5df1-4849-aeb4-cbfe3a96e674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:09.623356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eebc6d-5df1-4849-aeb4-cbfe3a96e674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:09.625200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eebc6d-5df1-4849-aeb4-cbfe3a96e674 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:09.642869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 608/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:09.644166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27192b6e-4839-4f77-bde5-02adaafa2c27 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:09.731035 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:09.765032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27192b6e-4839-4f77-bde5-02adaafa2c27 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:09.807105 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e6cc5-1d1b-4422-a3c7-c1d724908a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:09.807962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e6cc5-1d1b-4422-a3c7-c1d724908a63 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:09.808255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e6cc5-1d1b-4422-a3c7-c1d724908a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:09.808568 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e6cc5-1d1b-4422-a3c7-c1d724908a63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:09.816385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e6cc5-1d1b-4422-a3c7-c1d724908a63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:09.817869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e6cc5-1d1b-4422-a3c7-c1d724908a63 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-364e6cc5-1d1b-4422-a3c7-c1d724908a63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:23:09.832339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:09.833534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:09.834675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:23:09.834675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:23:09.834675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:23:09.834675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:23:09.834675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:23:09.834675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:23:09.834675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:23:09.834675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:09.834675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 609/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:23:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:09.839465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eebc6d-5df1-4849-aeb4-cbfe3a96e674 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:09.953396 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eebc6d-5df1-4849-aeb4-cbfe3a96e674 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:09.979393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:09.992778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1beeb-7d9b-44ab-bf35-ec54a543a466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:09.993449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1beeb-7d9b-44ab-bf35-ec54a543a466 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:09.993713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1beeb-7d9b-44ab-bf35-ec54a543a466 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:09.993990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1beeb-7d9b-44ab-bf35-ec54a543a466 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:09.994818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1beeb-7d9b-44ab-bf35-ec54a543a466 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:09.995191 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1beeb-7d9b-44ab-bf35-ec54a543a466 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:09.996630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1beeb-7d9b-44ab-bf35-ec54a543a466 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:10.010312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 609/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:23:10.017307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e6cc5-1d1b-4422-a3c7-c1d724908a63 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:10.137398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e6cc5-1d1b-4422-a3c7-c1d724908a63 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:10.178541 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:10.186029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91743-a7d7-43ff-b94c-2467f2911aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:10.186793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91743-a7d7-43ff-b94c-2467f2911aed None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:10.187212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91743-a7d7-43ff-b94c-2467f2911aed None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:10.188031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91743-a7d7-43ff-b94c-2467f2911aed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:10.189132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab91743-a7d7-43ff-b94c-2467f2911aed None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=705ff0e73e95499e873b1df2a24fefe8) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:10.189455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab91743-a7d7-43ff-b94c-2467f2911aed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:10.191154 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab91743-a7d7-43ff-b94c-2467f2911aed None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:10.197857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 610/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:23:10 2022] GET /identity/v3/role_assignments?user.id=705ff0e73e95499e873b1df2a24fefe8&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:23:12.399604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9403154-cf8b-4b7c-a70e-a49394f50ea0 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:12.400452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9403154-cf8b-4b7c-a70e-a49394f50ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:12.400452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9403154-cf8b-4b7c-a70e-a49394f50ea0 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:12.401627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 610/1220] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:12.407554 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3d281-cee7-4441-8eda-992d8f638eea None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:12.408126 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3d281-cee7-4441-8eda-992d8f638eea None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:12.408272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3d281-cee7-4441-8eda-992d8f638eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:12.427550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-aea3d281-cee7-4441-8eda-992d8f638eea None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:12.428497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-aea3d281-cee7-4441-8eda-992d8f638eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:12.503265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 611/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:23:12.526241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9403154-cf8b-4b7c-a70e-a49394f50ea0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:12.645771 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9403154-cf8b-4b7c-a70e-a49394f50ea0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:12.686723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b191a-e11e-4451-9f1c-16290be90c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:12.687648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b191a-e11e-4451-9f1c-16290be90c07 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:12.687936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b191a-e11e-4451-9f1c-16290be90c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:12.688219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b191a-e11e-4451-9f1c-16290be90c07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:12.703281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b191a-e11e-4451-9f1c-16290be90c07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-db0b191a-e11e-4451-9f1c-16290be90c07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:23:12.711276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:12.712553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:12.713474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 611/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:23:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:12.719440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3d281-cee7-4441-8eda-992d8f638eea None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:12.853063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3d281-cee7-4441-8eda-992d8f638eea None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:12.883809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:12.896994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10b206-f19f-4c0d-83c8-8cb0f05683b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:12.897733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10b206-f19f-4c0d-83c8-8cb0f05683b5 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:12.898058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10b206-f19f-4c0d-83c8-8cb0f05683b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:12.898400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10b206-f19f-4c0d-83c8-8cb0f05683b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:12.899319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10b206-f19f-4c0d-83c8-8cb0f05683b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:12.899665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10b206-f19f-4c0d-83c8-8cb0f05683b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:12.901248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10b206-f19f-4c0d-83c8-8cb0f05683b5 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:12.917534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 612/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:23:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:12.924360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b191a-e11e-4451-9f1c-16290be90c07 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:12.980544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:13.049169 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b191a-e11e-4451-9f1c-16290be90c07 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:13.109975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d5bf5-b712-4257-b370-9281a909cfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:13.110900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d5bf5-b712-4257-b370-9281a909cfbe None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:13.111191 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d5bf5-b712-4257-b370-9281a909cfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:13.111609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d5bf5-b712-4257-b370-9281a909cfbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:13.126651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d5bf5-b712-4257-b370-9281a909cfbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:13.128314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d5bf5-b712-4257-b370-9281a909cfbe None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-025d5bf5-b712-4257-b370-9281a909cfbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:13.152389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:13.153525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:13.161075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:13.161075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:13.161075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:13.161075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:13.161075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 612/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:13.161700 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10b206-f19f-4c0d-83c8-8cb0f05683b5 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:13.300741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10b206-f19f-4c0d-83c8-8cb0f05683b5 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:13.348550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd796042-a755-4cd6-baff-a709ec4a944f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:13.349503 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd796042-a755-4cd6-baff-a709ec4a944f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:13.350022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd796042-a755-4cd6-baff-a709ec4a944f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:13.350489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd796042-a755-4cd6-baff-a709ec4a944f None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:13.351526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd796042-a755-4cd6-baff-a709ec4a944f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:13.351924 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd796042-a755-4cd6-baff-a709ec4a944f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:13.353588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd796042-a755-4cd6-baff-a709ec4a944f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:13.372720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 613/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:13.381008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d5bf5-b712-4257-b370-9281a909cfbe None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:13.516269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:13.527179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d5bf5-b712-4257-b370-9281a909cfbe None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:13.574392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6edc8-abf0-4466-8e87-e9556d8fbd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:13.575839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6edc8-abf0-4466-8e87-e9556d8fbd6c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:13.576378 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6edc8-abf0-4466-8e87-e9556d8fbd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:13.576898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6edc8-abf0-4466-8e87-e9556d8fbd6c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:13.591654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6edc8-abf0-4466-8e87-e9556d8fbd6c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:13.594543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6edc8-abf0-4466-8e87-e9556d8fbd6c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-0aa6edc8-abf0-4466-8e87-e9556d8fbd6c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:23:13.615797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:13.617038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:13.618238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:13.618238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:13.618238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:23:13.618238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:23:13.618238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:23:13.618238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:23:13.618238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:23:13.618238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:23:13.618238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:23:13.618238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:13.618238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 613/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:13.626235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd796042-a755-4cd6-baff-a709ec4a944f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:13.691040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:13.774940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd796042-a755-4cd6-baff-a709ec4a944f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:13.831415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd73626-ce42-4e61-9bd4-a3e13b513513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:13.832039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd73626-ce42-4e61-9bd4-a3e13b513513 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:13.832324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd73626-ce42-4e61-9bd4-a3e13b513513 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:13.832662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd73626-ce42-4e61-9bd4-a3e13b513513 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:13.833541 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd73626-ce42-4e61-9bd4-a3e13b513513 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:13.833847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd73626-ce42-4e61-9bd4-a3e13b513513 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:13.835407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd73626-ce42-4e61-9bd4-a3e13b513513 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:13.857167 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 614/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jul 21 22:23:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:13.863663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6edc8-abf0-4466-8e87-e9556d8fbd6c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:13.994864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6edc8-abf0-4466-8e87-e9556d8fbd6c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:14.043168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cb4d7-57e3-44a2-a1b2-ad1cbd0666f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:14.044168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cb4d7-57e3-44a2-a1b2-ad1cbd0666f5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:14.044555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cb4d7-57e3-44a2-a1b2-ad1cbd0666f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:14.044837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cb4d7-57e3-44a2-a1b2-ad1cbd0666f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:14.056890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cb4d7-57e3-44a2-a1b2-ad1cbd0666f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:14.058814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cb4d7-57e3-44a2-a1b2-ad1cbd0666f5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-279cb4d7-57e3-44a2-a1b2-ad1cbd0666f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:14.084816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:14.090697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:14.091660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:14.091660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:14.091660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:14.091660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:14.091660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 614/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:14.091958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd73626-ce42-4e61-9bd4-a3e13b513513 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:14.245499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd73626-ce42-4e61-9bd4-a3e13b513513 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:14.248226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:14.293693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3401e9-2690-47cc-a41c-0e395595fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:14.294821 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3401e9-2690-47cc-a41c-0e395595fac6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:14.295326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3401e9-2690-47cc-a41c-0e395595fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:14.295779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3401e9-2690-47cc-a41c-0e395595fac6 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:14.297787 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3401e9-2690-47cc-a41c-0e395595fac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:14.298320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3401e9-2690-47cc-a41c-0e395595fac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:14.300705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3401e9-2690-47cc-a41c-0e395595fac6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:14.326570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 615/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:23:14.329364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cb4d7-57e3-44a2-a1b2-ad1cbd0666f5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:14.346790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:14.495586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cb4d7-57e3-44a2-a1b2-ad1cbd0666f5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:14.546501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09209d1-65d7-4748-9e23-7b0c904769e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:14.547722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09209d1-65d7-4748-9e23-7b0c904769e1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:14.547883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09209d1-65d7-4748-9e23-7b0c904769e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:14.548122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09209d1-65d7-4748-9e23-7b0c904769e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:14.556393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09209d1-65d7-4748-9e23-7b0c904769e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:14.558050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09209d1-65d7-4748-9e23-7b0c904769e1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e09209d1-65d7-4748-9e23-7b0c904769e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:23:14.574421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:14.575943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:14.578361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:23:14.578361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:23:14.578361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:23:14.578361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:23:14.578361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:23:14.578361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:23:14.578361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:23:14.578361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:14.578361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 615/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:23:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:14.583309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3401e9-2690-47cc-a41c-0e395595fac6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:14.717319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3401e9-2690-47cc-a41c-0e395595fac6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:14.760756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2852bf-ce00-47ca-b5db-ac3b991f6414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:14.761453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2852bf-ce00-47ca-b5db-ac3b991f6414 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:14.761938 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2852bf-ce00-47ca-b5db-ac3b991f6414 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:14.762338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2852bf-ce00-47ca-b5db-ac3b991f6414 None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:14.763760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2852bf-ce00-47ca-b5db-ac3b991f6414 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:14.764280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2852bf-ce00-47ca-b5db-ac3b991f6414 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:14.767270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2852bf-ce00-47ca-b5db-ac3b991f6414 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:14.782865 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 616/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:23:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:14.790373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09209d1-65d7-4748-9e23-7b0c904769e1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:14.879249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:14.923679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09209d1-65d7-4748-9e23-7b0c904769e1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:14.964528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415df27-819b-4ee8-8595-ac3ea29087e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:14.965443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415df27-819b-4ee8-8595-ac3ea29087e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:14.965724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415df27-819b-4ee8-8595-ac3ea29087e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:14.965996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415df27-819b-4ee8-8595-ac3ea29087e4 None admin] PATH_INFO: `/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/705ff0e73e95499e873b1df2a24fefe8/roles/ca5450234bd44b46a10185c4c8a0647f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:14.983441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415df27-819b-4ee8-8595-ac3ea29087e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0638fe2ce7a40299f2d654f58ed535c, user_id=705ff0e73e95499e873b1df2a24fefe8, role_id=ca5450234bd44b46a10185c4c8a0647f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:14.984936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415df27-819b-4ee8-8595-ac3ea29087e4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:15.002142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 616/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:23:14 2022] PUT /identity/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/705ff0e73e95499e873b1df2a24fefe8/roles/ca5450234bd44b46a10185c4c8a0647f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:23:17.243598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d21f-0113-45f4-92c0-8d4ae695d949 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:17.244398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d21f-0113-45f4-92c0-8d4ae695d949 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:17.244398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d21f-0113-45f4-92c0-8d4ae695d949 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:17.245713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 617/1233] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:17.251287 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df206f1-e113-4473-9e81-f89fc96bb447 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:17.251788 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df206f1-e113-4473-9e81-f89fc96bb447 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:17.251901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df206f1-e113-4473-9e81-f89fc96bb447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:17.270536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3df206f1-e113-4473-9e81-f89fc96bb447 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:17.271727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3df206f1-e113-4473-9e81-f89fc96bb447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:17.353123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 617/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:17.380968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51d21f-0113-45f4-92c0-8d4ae695d949 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:17.416428 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:17.541691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51d21f-0113-45f4-92c0-8d4ae695d949 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:17.585728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44028b6-cce5-4134-b5ed-0d47ee0ef505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:17.586543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44028b6-cce5-4134-b5ed-0d47ee0ef505 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:17.586834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44028b6-cce5-4134-b5ed-0d47ee0ef505 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:17.587113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44028b6-cce5-4134-b5ed-0d47ee0ef505 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:17.609531 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44028b6-cce5-4134-b5ed-0d47ee0ef505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-c44028b6-cce5-4134-b5ed-0d47ee0ef505 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:23:17.618830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:17.624697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:17.625674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 618/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Thu Jul 21 22:23:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:17.633550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df206f1-e113-4473-9e81-f89fc96bb447 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:17.688948 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:17.781533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df206f1-e113-4473-9e81-f89fc96bb447 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:17.846191 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b45195-0205-49ff-9b8c-d23b7cd551f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:17.847257 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b45195-0205-49ff-9b8c-d23b7cd551f4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:17.847697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b45195-0205-49ff-9b8c-d23b7cd551f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:17.848174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b45195-0205-49ff-9b8c-d23b7cd551f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:17.849126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b45195-0205-49ff-9b8c-d23b7cd551f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:17.849596 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b45195-0205-49ff-9b8c-d23b7cd551f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:17.852688 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b45195-0205-49ff-9b8c-d23b7cd551f4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:17.866115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 618/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Thu Jul 21 22:23:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:17.873054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44028b6-cce5-4134-b5ed-0d47ee0ef505 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:18.000914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:18.021101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44028b6-cce5-4134-b5ed-0d47ee0ef505 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:18.064428 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e5933-28a4-4c53-a8c6-12c36c6a51fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:18.065326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e5933-28a4-4c53-a8c6-12c36c6a51fd None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:18.065638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e5933-28a4-4c53-a8c6-12c36c6a51fd None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:18.066040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e5933-28a4-4c53-a8c6-12c36c6a51fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:18.079088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e5933-28a4-4c53-a8c6-12c36c6a51fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:18.080757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e5933-28a4-4c53-a8c6-12c36c6a51fd None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-460e5933-28a4-4c53-a8c6-12c36c6a51fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:18.101499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:18.102540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:18.109225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:18.109225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:18.109225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:18.109225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:18.109225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 619/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:18.109533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b45195-0205-49ff-9b8c-d23b7cd551f4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:18.290869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b45195-0205-49ff-9b8c-d23b7cd551f4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:18.303258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:18.337700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ad94f-c0cc-4561-bc08-355d1dab6b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:18.338387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ad94f-c0cc-4561-bc08-355d1dab6b2d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:18.338668 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ad94f-c0cc-4561-bc08-355d1dab6b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:18.338945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ad94f-c0cc-4561-bc08-355d1dab6b2d None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:18.339732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ad94f-c0cc-4561-bc08-355d1dab6b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:18.340119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ad94f-c0cc-4561-bc08-355d1dab6b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:18.341741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ad94f-c0cc-4561-bc08-355d1dab6b2d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:18.359365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 619/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:23:18.367080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e5933-28a4-4c53-a8c6-12c36c6a51fd None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:18.504016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e5933-28a4-4c53-a8c6-12c36c6a51fd None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:18.556583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368be7a1-5b13-4526-8018-025ca6574dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:18.557648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368be7a1-5b13-4526-8018-025ca6574dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:18.557957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368be7a1-5b13-4526-8018-025ca6574dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:18.558138 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368be7a1-5b13-4526-8018-025ca6574dd7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:18.562878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:18.569637 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368be7a1-5b13-4526-8018-025ca6574dd7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:18.571727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368be7a1-5b13-4526-8018-025ca6574dd7 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-368be7a1-5b13-4526-8018-025ca6574dd7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:23:18.599304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:18.600961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:18.603380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:18.603380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:18.603380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:23:18.603380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:23:18.603380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:23:18.603380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:23:18.603380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:23:18.603380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:23:18.603380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:23:18.603380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:18.603380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 620/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:18.608123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ad94f-c0cc-4561-bc08-355d1dab6b2d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:18.776241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ad94f-c0cc-4561-bc08-355d1dab6b2d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:18.829505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dac21-a961-4b4d-8fec-4ca3cc4b15ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:18.830263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dac21-a961-4b4d-8fec-4ca3cc4b15ad None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:18.830562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dac21-a961-4b4d-8fec-4ca3cc4b15ad None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:18.830877 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dac21-a961-4b4d-8fec-4ca3cc4b15ad None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:18.831820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752dac21-a961-4b4d-8fec-4ca3cc4b15ad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:18.832168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752dac21-a961-4b4d-8fec-4ca3cc4b15ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:18.833807 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752dac21-a961-4b4d-8fec-4ca3cc4b15ad None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:18.856732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 620/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jul 21 22:23:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:23:18.863021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368be7a1-5b13-4526-8018-025ca6574dd7 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:19.028229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368be7a1-5b13-4526-8018-025ca6574dd7 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:19.076071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40416d35-480b-47fe-98f7-b2913c4937e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:19.078034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40416d35-480b-47fe-98f7-b2913c4937e7 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:19.078390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40416d35-480b-47fe-98f7-b2913c4937e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:19.078798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40416d35-480b-47fe-98f7-b2913c4937e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:19.189434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:19.207921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40416d35-480b-47fe-98f7-b2913c4937e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:19.210537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40416d35-480b-47fe-98f7-b2913c4937e7 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:19.359737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 621/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:23:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:23:19.368402 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dac21-a961-4b4d-8fec-4ca3cc4b15ad None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:19.410108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:19.533887 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dac21-a961-4b4d-8fec-4ca3cc4b15ad None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:19.588787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05620db0-b422-43d7-bb09-98ca54efa827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:19.590997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05620db0-b422-43d7-bb09-98ca54efa827 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:19.591676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05620db0-b422-43d7-bb09-98ca54efa827 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:19.592125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05620db0-b422-43d7-bb09-98ca54efa827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:19.612766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05620db0-b422-43d7-bb09-98ca54efa827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:19.614909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05620db0-b422-43d7-bb09-98ca54efa827 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-05620db0-b422-43d7-bb09-98ca54efa827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:19.636198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:19.637537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:19.639097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:19.639097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:19.639097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:19.639097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:19.639097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 621/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:23:19.645262 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40416d35-480b-47fe-98f7-b2913c4937e7 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:19.775686 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:19.782851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40416d35-480b-47fe-98f7-b2913c4937e7 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:19.828212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0e8df-4f9b-43aa-94a5-08ea8d80fe45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:19.828954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0e8df-4f9b-43aa-94a5-08ea8d80fe45 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:19.829473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0e8df-4f9b-43aa-94a5-08ea8d80fe45 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:19.829473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0e8df-4f9b-43aa-94a5-08ea8d80fe45 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:19.830395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0e8df-4f9b-43aa-94a5-08ea8d80fe45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:19.830730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0e8df-4f9b-43aa-94a5-08ea8d80fe45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:19.832242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0e8df-4f9b-43aa-94a5-08ea8d80fe45 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:19.847179 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 622/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:19.855297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05620db0-b422-43d7-bb09-98ca54efa827 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:19.977463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:20.010323 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05620db0-b422-43d7-bb09-98ca54efa827 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:20.054652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30c40f-e051-4b2a-8e41-c268e092e61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:20.055458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c40f-e051-4b2a-8e41-c268e092e61d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:20.055743 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c40f-e051-4b2a-8e41-c268e092e61d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:20.056023 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c40f-e051-4b2a-8e41-c268e092e61d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:20.065594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30c40f-e051-4b2a-8e41-c268e092e61d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:20.067342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30c40f-e051-4b2a-8e41-c268e092e61d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-5f30c40f-e051-4b2a-8e41-c268e092e61d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:23:20.087704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:20.089308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:20.091146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:23:20.091146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:23:20.091146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:23:20.091146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:23:20.091146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:23:20.091146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:23:20.091146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:23:20.091146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:20.091146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 622/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:23:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:23:20.097426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0e8df-4f9b-43aa-94a5-08ea8d80fe45 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:20.258785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0e8df-4f9b-43aa-94a5-08ea8d80fe45 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:20.306353 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f9c9e-e3f7-432b-b4d6-28db93843c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:20.307035 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f9c9e-e3f7-432b-b4d6-28db93843c1b None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:20.307335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f9c9e-e3f7-432b-b4d6-28db93843c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:20.307611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f9c9e-e3f7-432b-b4d6-28db93843c1b None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:20.308399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f9c9e-e3f7-432b-b4d6-28db93843c1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:20.308741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f9c9e-e3f7-432b-b4d6-28db93843c1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:20.310266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f9c9e-e3f7-432b-b4d6-28db93843c1b None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:20.326812 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 623/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:23:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:23:20.336979 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30c40f-e051-4b2a-8e41-c268e092e61d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:20.488261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30c40f-e051-4b2a-8e41-c268e092e61d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:20.545729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8876ca6-60e4-46f6-9443-108991325719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:20.546759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8876ca6-60e4-46f6-9443-108991325719 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:20.547117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8876ca6-60e4-46f6-9443-108991325719 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:20.547510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8876ca6-60e4-46f6-9443-108991325719 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:20.548664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8876ca6-60e4-46f6-9443-108991325719 None admin] RBAC: Adding query filter params (role.id=ca5450234bd44b46a10185c4c8a0647f, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=705ff0e73e95499e873b1df2a24fefe8) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:20.549201 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8876ca6-60e4-46f6-9443-108991325719 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:20.551400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8876ca6-60e4-46f6-9443-108991325719 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:20.559944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 623/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:23:20 2022] GET /identity/v3/role_assignments?user.id=705ff0e73e95499e873b1df2a24fefe8&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=ca5450234bd44b46a10185c4c8a0647f => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:22.952965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea47b-e9cf-4479-a6db-79d9e0513dbc None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:22.953606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea47b-e9cf-4479-a6db-79d9e0513dbc None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:22.954050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea47b-e9cf-4479-a6db-79d9e0513dbc None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:22.956229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 624/1247] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:23:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:22.966489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f06c2-c92a-47b3-a655-1aa94210c690 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:22.967031 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f06c2-c92a-47b3-a655-1aa94210c690 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:22.968158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f06c2-c92a-47b3-a655-1aa94210c690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:22.977381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:22.999316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-718f06c2-c92a-47b3-a655-1aa94210c690 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:23.000712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-718f06c2-c92a-47b3-a655-1aa94210c690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:23.106828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 624/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:23.137412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ea47b-e9cf-4479-a6db-79d9e0513dbc None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:23.165163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:23.323141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ea47b-e9cf-4479-a6db-79d9e0513dbc None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:23.386848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f0ca2-0731-4c04-bc1c-709705cc9af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:23.388150 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f0ca2-0731-4c04-bc1c-709705cc9af1 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:23.388565 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f0ca2-0731-4c04-bc1c-709705cc9af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:23.389002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f0ca2-0731-4c04-bc1c-709705cc9af1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:23.402610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f0ca2-0731-4c04-bc1c-709705cc9af1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-9b4f0ca2-0731-4c04-bc1c-709705cc9af1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:23:23.411496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:23.413032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:23.414942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 625/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:23:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:23.423222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f06c2-c92a-47b3-a655-1aa94210c690 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:23.583807 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f06c2-c92a-47b3-a655-1aa94210c690 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:23.638596 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:23.647357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b837d-716c-499b-8ef1-361de259b73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:23.648258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b837d-716c-499b-8ef1-361de259b73e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:23.648637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b837d-716c-499b-8ef1-361de259b73e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:23.649060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b837d-716c-499b-8ef1-361de259b73e None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:23.650146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b837d-716c-499b-8ef1-361de259b73e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:23.650606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b837d-716c-499b-8ef1-361de259b73e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:23.652386 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b837d-716c-499b-8ef1-361de259b73e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:23.664296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 625/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:23:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:23.673001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f0ca2-0731-4c04-bc1c-709705cc9af1 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:23.804746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:23.840320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f0ca2-0731-4c04-bc1c-709705cc9af1 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:23.882151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c58fc-78c8-4e1a-adcc-064d4639dbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:23.882875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c58fc-78c8-4e1a-adcc-064d4639dbeb None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:23.883160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c58fc-78c8-4e1a-adcc-064d4639dbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:23.883442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c58fc-78c8-4e1a-adcc-064d4639dbeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:23.897642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c58fc-78c8-4e1a-adcc-064d4639dbeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:23.899128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c58fc-78c8-4e1a-adcc-064d4639dbeb None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-e98c58fc-78c8-4e1a-adcc-064d4639dbeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:23.921497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:23.922899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:23.923978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:23.923978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:23.923978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:23.923978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:23.923978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 626/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:23.930317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b837d-716c-499b-8ef1-361de259b73e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:24.077954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b837d-716c-499b-8ef1-361de259b73e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:24.136543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5a55b-fe2e-4881-9dbe-16d6734a18d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:24.137632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5a55b-fe2e-4881-9dbe-16d6734a18d5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:24.138416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5a55b-fe2e-4881-9dbe-16d6734a18d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:24.140440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5a55b-fe2e-4881-9dbe-16d6734a18d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:24.141694 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5a55b-fe2e-4881-9dbe-16d6734a18d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:24.142014 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5a55b-fe2e-4881-9dbe-16d6734a18d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:24.143908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5a55b-fe2e-4881-9dbe-16d6734a18d5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:24.162754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 626/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:24.171850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c58fc-78c8-4e1a-adcc-064d4639dbeb None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:24.341679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c58fc-78c8-4e1a-adcc-064d4639dbeb None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:24.376969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:24.390974 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328afe32-41bf-44fb-987c-0d8a583c52a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:24.391823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328afe32-41bf-44fb-987c-0d8a583c52a9 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:24.392242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328afe32-41bf-44fb-987c-0d8a583c52a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:24.392554 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328afe32-41bf-44fb-987c-0d8a583c52a9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:24.401670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328afe32-41bf-44fb-987c-0d8a583c52a9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:24.403397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328afe32-41bf-44fb-987c-0d8a583c52a9 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-328afe32-41bf-44fb-987c-0d8a583c52a9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:23:24.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:24.435161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:24.437063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:24.437063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:24.437063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:23:24.437063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:23:24.437063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:23:24.437063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:23:24.437063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:23:24.437063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:23:24.437063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:23:24.437063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:24.437063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 627/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:24.443451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5a55b-fe2e-4881-9dbe-16d6734a18d5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:24.521995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:24.591909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5a55b-fe2e-4881-9dbe-16d6734a18d5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:24.645206 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d98980-5eea-4935-a7f7-b4ca3ffa14af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:24.645917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d98980-5eea-4935-a7f7-b4ca3ffa14af None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:24.646261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d98980-5eea-4935-a7f7-b4ca3ffa14af None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:24.646591 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d98980-5eea-4935-a7f7-b4ca3ffa14af None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:24.647507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d98980-5eea-4935-a7f7-b4ca3ffa14af None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:24.647873 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d98980-5eea-4935-a7f7-b4ca3ffa14af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:24.649449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d98980-5eea-4935-a7f7-b4ca3ffa14af None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:24.672285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 627/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jul 21 22:23:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:24.678956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328afe32-41bf-44fb-987c-0d8a583c52a9 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:24.823898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328afe32-41bf-44fb-987c-0d8a583c52a9 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:24.893720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960022af-0b35-4ab7-a173-019cbb25f0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:24.894421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960022af-0b35-4ab7-a173-019cbb25f0ba None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:24.894758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960022af-0b35-4ab7-a173-019cbb25f0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:24.894988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960022af-0b35-4ab7-a173-019cbb25f0ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:24.995128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:25.011465 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960022af-0b35-4ab7-a173-019cbb25f0ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:25.013783 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960022af-0b35-4ab7-a173-019cbb25f0ba None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:25.164322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 628/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:23:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:23:25.173755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d98980-5eea-4935-a7f7-b4ca3ffa14af None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:25.315462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d98980-5eea-4935-a7f7-b4ca3ffa14af None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:25.345359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:25.399452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a677fb-c69d-49b4-823c-beaccd37595f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:25.400289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a677fb-c69d-49b4-823c-beaccd37595f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:25.400612 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a677fb-c69d-49b4-823c-beaccd37595f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:25.400900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a677fb-c69d-49b4-823c-beaccd37595f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:25.418940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a677fb-c69d-49b4-823c-beaccd37595f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:25.421617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a677fb-c69d-49b4-823c-beaccd37595f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-58a677fb-c69d-49b4-823c-beaccd37595f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:25.444276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:25.445821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:25.446937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:25.446937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:25.446937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:25.446937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:25.446937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 628/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:25.452764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960022af-0b35-4ab7-a173-019cbb25f0ba None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:25.584043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:25.600016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960022af-0b35-4ab7-a173-019cbb25f0ba None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:25.642914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb85fe-b00c-415a-afbb-5c59a9d2b2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:25.643659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb85fe-b00c-415a-afbb-5c59a9d2b2ae None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:25.644018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb85fe-b00c-415a-afbb-5c59a9d2b2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:25.644351 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb85fe-b00c-415a-afbb-5c59a9d2b2ae None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:25.645263 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb85fe-b00c-415a-afbb-5c59a9d2b2ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:25.645574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb85fe-b00c-415a-afbb-5c59a9d2b2ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:25.647026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb85fe-b00c-415a-afbb-5c59a9d2b2ae None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:25.660557 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 629/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:23:25.669113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a677fb-c69d-49b4-823c-beaccd37595f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:25.824861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a677fb-c69d-49b4-823c-beaccd37595f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:25.867029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707bef6-183f-47a3-abd6-1dccd6c405ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:25.867787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707bef6-183f-47a3-abd6-1dccd6c405ea None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:25.868106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707bef6-183f-47a3-abd6-1dccd6c405ea None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:25.868558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707bef6-183f-47a3-abd6-1dccd6c405ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:25.877518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707bef6-183f-47a3-abd6-1dccd6c405ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:25.879086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707bef6-183f-47a3-abd6-1dccd6c405ea None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-1707bef6-183f-47a3-abd6-1dccd6c405ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:23:25.895964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:25.897248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:25.899086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:23:25.899086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:23:25.899086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:23:25.899086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:23:25.899086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:23:25.899086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:23:25.899086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:23:25.899086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:25.899086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 629/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:23:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:25.906264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb85fe-b00c-415a-afbb-5c59a9d2b2ae None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:26.040706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb85fe-b00c-415a-afbb-5c59a9d2b2ae None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:26.085212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbadaa-0f18-44e8-b31f-ca68d75e2c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:26.086237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbadaa-0f18-44e8-b31f-ca68d75e2c5a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:26.086756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbadaa-0f18-44e8-b31f-ca68d75e2c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:26.087240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbadaa-0f18-44e8-b31f-ca68d75e2c5a None admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:26.088616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbadaa-0f18-44e8-b31f-ca68d75e2c5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:26.089115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbadaa-0f18-44e8-b31f-ca68d75e2c5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:26.091508 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbadaa-0f18-44e8-b31f-ca68d75e2c5a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:26.107939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 630/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:26.115085 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707bef6-183f-47a3-abd6-1dccd6c405ea None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:26.156356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:26.258951 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707bef6-183f-47a3-abd6-1dccd6c405ea None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:26.323764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d250745-16f2-4d47-889d-f4d9329175cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:26.324437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d250745-16f2-4d47-889d-f4d9329175cf None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:26.324746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d250745-16f2-4d47-889d-f4d9329175cf None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:26.325102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d250745-16f2-4d47-889d-f4d9329175cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:26.325874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d250745-16f2-4d47-889d-f4d9329175cf None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=705ff0e73e95499e873b1df2a24fefe8) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:26.326329 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d250745-16f2-4d47-889d-f4d9329175cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:26.329162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d250745-16f2-4d47-889d-f4d9329175cf None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:26.337123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 630/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:23:26 2022] GET /identity/v3/role_assignments?user.id=705ff0e73e95499e873b1df2a24fefe8&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=f957e09d047b47cfa0891d25b4181263 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:28.507725 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb79c5-2cd8-4e48-8183-24f2a7356e8f None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:28.508270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb79c5-2cd8-4e48-8183-24f2a7356e8f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:28.508380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb79c5-2cd8-4e48-8183-24f2a7356e8f None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:28.509685 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 631/1261] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:28.514964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e367b-400a-46de-8355-99358ee19aa9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:28.515430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e367b-400a-46de-8355-99358ee19aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:28.515538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e367b-400a-46de-8355-99358ee19aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:28.532648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ed1e367b-400a-46de-8355-99358ee19aa9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:28.533845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ed1e367b-400a-46de-8355-99358ee19aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:28.599628 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 631/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:28.622526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb79c5-2cd8-4e48-8183-24f2a7356e8f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:28.633604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:28.727655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb79c5-2cd8-4e48-8183-24f2a7356e8f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:28.761883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8aefa-5aff-44d2-aeca-2ad05b382185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:28.762937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8aefa-5aff-44d2-aeca-2ad05b382185 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:28.763226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8aefa-5aff-44d2-aeca-2ad05b382185 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:28.763503 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8aefa-5aff-44d2-aeca-2ad05b382185 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:28.774025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8aefa-5aff-44d2-aeca-2ad05b382185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-63c8aefa-5aff-44d2-aeca-2ad05b382185 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:23:28.782407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:28.783619 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:28.784710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 632/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:23:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:28.789752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e367b-400a-46de-8355-99358ee19aa9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:28.808235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:28.889177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e367b-400a-46de-8355-99358ee19aa9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:28.923122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65e193-f7b9-48ee-b056-a1a3d4b49f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:28.923785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65e193-f7b9-48ee-b056-a1a3d4b49f9e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:28.924067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65e193-f7b9-48ee-b056-a1a3d4b49f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:28.924341 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65e193-f7b9-48ee-b056-a1a3d4b49f9e None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:28.925116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65e193-f7b9-48ee-b056-a1a3d4b49f9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:28.925424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65e193-f7b9-48ee-b056-a1a3d4b49f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:28.926796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65e193-f7b9-48ee-b056-a1a3d4b49f9e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:28.934829 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 632/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:23:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:28.940740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8aefa-5aff-44d2-aeca-2ad05b382185 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:29.006992 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:29.040527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8aefa-5aff-44d2-aeca-2ad05b382185 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:29.073727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b9d29-03de-47a8-8145-592ebac6e8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:29.074522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b9d29-03de-47a8-8145-592ebac6e8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:29.074810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b9d29-03de-47a8-8145-592ebac6e8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:29.075092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b9d29-03de-47a8-8145-592ebac6e8d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:29.085844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b9d29-03de-47a8-8145-592ebac6e8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:29.087288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b9d29-03de-47a8-8145-592ebac6e8d0 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-c08b9d29-03de-47a8-8145-592ebac6e8d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:29.104555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:29.105692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:29.106837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:29.106837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:29.106837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:29.106837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:29.106837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 633/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:29.112448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65e193-f7b9-48ee-b056-a1a3d4b49f9e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:29.203548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:29.222601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65e193-f7b9-48ee-b056-a1a3d4b49f9e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:29.256880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b1bca-127c-452b-9104-06c06b25b272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:29.257855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b1bca-127c-452b-9104-06c06b25b272 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:29.258224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b1bca-127c-452b-9104-06c06b25b272 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:29.258503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b1bca-127c-452b-9104-06c06b25b272 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:29.259459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b1bca-127c-452b-9104-06c06b25b272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:29.259795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b1bca-127c-452b-9104-06c06b25b272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:29.261607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b1bca-127c-452b-9104-06c06b25b272 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:29.276816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 633/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:23:29.284359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b9d29-03de-47a8-8145-592ebac6e8d0 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:29.398777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b9d29-03de-47a8-8145-592ebac6e8d0 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:29.419118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:29.434041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633304c5-a7d4-4ab3-8308-73188f74351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:29.434857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633304c5-a7d4-4ab3-8308-73188f74351c None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:29.435143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633304c5-a7d4-4ab3-8308-73188f74351c None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:29.435497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633304c5-a7d4-4ab3-8308-73188f74351c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:29.450973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633304c5-a7d4-4ab3-8308-73188f74351c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:29.453090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633304c5-a7d4-4ab3-8308-73188f74351c None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-633304c5-a7d4-4ab3-8308-73188f74351c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:23:29.469089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:29.470315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:29.471571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:29.471571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:29.471571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:23:29.471571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:23:29.471571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:23:29.471571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:23:29.471571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:23:29.471571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:23:29.471571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:23:29.471571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:29.471571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 634/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:29.476409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b1bca-127c-452b-9104-06c06b25b272 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:29.577570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b1bca-127c-452b-9104-06c06b25b272 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:29.615658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea6258-d6f7-4660-b152-cfaa81341794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:29.616328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea6258-d6f7-4660-b152-cfaa81341794 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:29.616641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea6258-d6f7-4660-b152-cfaa81341794 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:29.616913 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea6258-d6f7-4660-b152-cfaa81341794 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:29.617749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea6258-d6f7-4660-b152-cfaa81341794 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:29.618055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea6258-d6f7-4660-b152-cfaa81341794 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:29.619492 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea6258-d6f7-4660-b152-cfaa81341794 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:29.628856 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:29.639018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 634/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jul 21 22:23:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:29.644823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633304c5-a7d4-4ab3-8308-73188f74351c None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:29.756879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633304c5-a7d4-4ab3-8308-73188f74351c None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:29.790840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335428e-94cc-47fe-8d84-640af0f09e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:29.791633 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335428e-94cc-47fe-8d84-640af0f09e17 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:29.791916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335428e-94cc-47fe-8d84-640af0f09e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:29.792197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335428e-94cc-47fe-8d84-640af0f09e17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:29.802963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335428e-94cc-47fe-8d84-640af0f09e17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:29.804391 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335428e-94cc-47fe-8d84-640af0f09e17 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-f335428e-94cc-47fe-8d84-640af0f09e17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:29.821421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:29.822519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:29.823715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:29.823715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:29.823715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:29.823715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:29.823715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 635/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:29.829275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea6258-d6f7-4660-b152-cfaa81341794 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:29.926818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea6258-d6f7-4660-b152-cfaa81341794 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:29.961666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcbacb-cffb-41e7-af47-be570516f161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:29.963259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcbacb-cffb-41e7-af47-be570516f161 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:29.963539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcbacb-cffb-41e7-af47-be570516f161 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:29.963808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcbacb-cffb-41e7-af47-be570516f161 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:29.964553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcbacb-cffb-41e7-af47-be570516f161 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:29.964858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcbacb-cffb-41e7-af47-be570516f161 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:29.967240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcbacb-cffb-41e7-af47-be570516f161 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:29.979199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 635/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:29.985323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335428e-94cc-47fe-8d84-640af0f09e17 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:30.020651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:30.094879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335428e-94cc-47fe-8d84-640af0f09e17 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:30.129723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5b230-e7a4-4098-9ba3-cb02086bd593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:30.130523 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5b230-e7a4-4098-9ba3-cb02086bd593 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:30.130974 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5b230-e7a4-4098-9ba3-cb02086bd593 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:30.131153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5b230-e7a4-4098-9ba3-cb02086bd593 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:30.138365 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5b230-e7a4-4098-9ba3-cb02086bd593 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:30.140402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5b230-e7a4-4098-9ba3-cb02086bd593 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-bdc5b230-e7a4-4098-9ba3-cb02086bd593 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:23:30.155210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:30.156878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:30.158148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:23:30.158148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:23:30.158148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:23:30.158148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:23:30.158148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:23:30.158148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:23:30.158148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:23:30.158148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:30.158148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 636/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:23:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:30.162230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcbacb-cffb-41e7-af47-be570516f161 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:30.220127 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:30.273746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcbacb-cffb-41e7-af47-be570516f161 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:30.317847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a052c52-ca07-408e-a52b-783ea45d02a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:30.318694 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a052c52-ca07-408e-a52b-783ea45d02a8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:30.319030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a052c52-ca07-408e-a52b-783ea45d02a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:30.319372 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a052c52-ca07-408e-a52b-783ea45d02a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:30.320678 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a052c52-ca07-408e-a52b-783ea45d02a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:30.321152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a052c52-ca07-408e-a52b-783ea45d02a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:30.323521 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a052c52-ca07-408e-a52b-783ea45d02a8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:30.336784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 636/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:23:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:30.342999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5b230-e7a4-4098-9ba3-cb02086bd593 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:30.432544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:30.442198 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5b230-e7a4-4098-9ba3-cb02086bd593 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:30.474880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047b526-ec14-40bb-a68a-3310b5aae071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:30.475765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047b526-ec14-40bb-a68a-3310b5aae071 None admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:30.476046 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047b526-ec14-40bb-a68a-3310b5aae071 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:30.476318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047b526-ec14-40bb-a68a-3310b5aae071 None admin] PATH_INFO: `/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/705ff0e73e95499e873b1df2a24fefe8/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:30.478294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047b526-ec14-40bb-a68a-3310b5aae071 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0638fe2ce7a40299f2d654f58ed535c, user_id=705ff0e73e95499e873b1df2a24fefe8, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:30.479717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047b526-ec14-40bb-a68a-3310b5aae071 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:30.493079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 637/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Thu Jul 21 22:23:30 2022] PUT /identity/v3/projects/b0638fe2ce7a40299f2d654f58ed535c/users/705ff0e73e95499e873b1df2a24fefe8/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:23:32.593932 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c73c8a-6a61-44a4-8760-600e54db40bd None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:32.594497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c73c8a-6a61-44a4-8760-600e54db40bd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:32.594497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c73c8a-6a61-44a4-8760-600e54db40bd None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:32.595852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 637/1274] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:32.601119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5635b-7c17-44a9-849e-ee79aad99624 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:32.601540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5635b-7c17-44a9-849e-ee79aad99624 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:32.601683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5635b-7c17-44a9-849e-ee79aad99624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:32.618444 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-6df5635b-7c17-44a9-849e-ee79aad99624 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:32.619610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6df5635b-7c17-44a9-849e-ee79aad99624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:32.687155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 638/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:32.713631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c73c8a-6a61-44a4-8760-600e54db40bd None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:32.815449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c73c8a-6a61-44a4-8760-600e54db40bd None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:32.832177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:32.849647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e0c05-5c8b-4374-8614-155020140f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:32.850436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e0c05-5c8b-4374-8614-155020140f82 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:32.850719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e0c05-5c8b-4374-8614-155020140f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:32.850997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e0c05-5c8b-4374-8614-155020140f82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:32.861522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e0c05-5c8b-4374-8614-155020140f82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-5e9e0c05-5c8b-4374-8614-155020140f82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:23:32.869128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:32.870215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:32.871315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 638/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Thu Jul 21 22:23:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:32.876557 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5635b-7c17-44a9-849e-ee79aad99624 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:32.957866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:32.977369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5635b-7c17-44a9-849e-ee79aad99624 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:33.011108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de1e32-8055-4e48-a22c-1f1428d9f19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:33.011777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de1e32-8055-4e48-a22c-1f1428d9f19e None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:33.012065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de1e32-8055-4e48-a22c-1f1428d9f19e None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:33.012350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de1e32-8055-4e48-a22c-1f1428d9f19e None admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:33.014086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de1e32-8055-4e48-a22c-1f1428d9f19e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:33.014402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de1e32-8055-4e48-a22c-1f1428d9f19e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:33.015777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de1e32-8055-4e48-a22c-1f1428d9f19e None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:33.023945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 639/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Thu Jul 21 22:23:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:33.029815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e0c05-5c8b-4374-8614-155020140f82 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:33.129315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e0c05-5c8b-4374-8614-155020140f82 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:33.170101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e78ca-15cf-4dc4-9283-919aeb67f205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:33.171071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e78ca-15cf-4dc4-9283-919aeb67f205 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:33.171493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e78ca-15cf-4dc4-9283-919aeb67f205 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:33.171861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e78ca-15cf-4dc4-9283-919aeb67f205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:33.185918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e78ca-15cf-4dc4-9283-919aeb67f205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:33.187836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e78ca-15cf-4dc4-9283-919aeb67f205 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-a73e78ca-15cf-4dc4-9283-919aeb67f205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:33.207078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:33.208280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:33.209446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:33.209446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:33.209446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:33.209446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:33.209446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 639/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:33.214056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de1e32-8055-4e48-a22c-1f1428d9f19e None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:33.320548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de1e32-8055-4e48-a22c-1f1428d9f19e None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:33.353775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdb155-13e7-4a4a-a143-7c75d388b6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:33.354437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdb155-13e7-4a4a-a143-7c75d388b6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:33.354724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdb155-13e7-4a4a-a143-7c75d388b6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:33.355002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdb155-13e7-4a4a-a143-7c75d388b6e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:33.355730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdb155-13e7-4a4a-a143-7c75d388b6e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:33.356036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdb155-13e7-4a4a-a143-7c75d388b6e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:33.358479 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdb155-13e7-4a4a-a143-7c75d388b6e1 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:33.370011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 640/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:33.376186 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e78ca-15cf-4dc4-9283-919aeb67f205 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:33.407460 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:33.476763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e78ca-15cf-4dc4-9283-919aeb67f205 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:33.509899 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4542d-8994-4e1e-9b08-8422e352d4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:33.510692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4542d-8994-4e1e-9b08-8422e352d4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:33.510974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4542d-8994-4e1e-9b08-8422e352d4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:33.511247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4542d-8994-4e1e-9b08-8422e352d4e5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:33.518936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4542d-8994-4e1e-9b08-8422e352d4e5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:33.520370 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4542d-8994-4e1e-9b08-8422e352d4e5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-6ec4542d-8994-4e1e-9b08-8422e352d4e5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:23:33.535032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:33.536190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:33.537412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:33.537412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:33.537412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:23:33.537412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:23:33.537412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:23:33.537412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:23:33.537412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:23:33.537412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:23:33.537412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:23:33.537412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:33.537412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 640/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:33.542060 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdb155-13e7-4a4a-a143-7c75d388b6e1 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:33.546933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:33.648555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdb155-13e7-4a4a-a143-7c75d388b6e1 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:33.682917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a2d08-c493-4a79-8150-8ea4a3167604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:33.683592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a2d08-c493-4a79-8150-8ea4a3167604 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:33.683880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a2d08-c493-4a79-8150-8ea4a3167604 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:33.684161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a2d08-c493-4a79-8150-8ea4a3167604 None admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:33.685628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a2d08-c493-4a79-8150-8ea4a3167604 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:33.688998 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a2d08-c493-4a79-8150-8ea4a3167604 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:33.690935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a2d08-c493-4a79-8150-8ea4a3167604 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:33.709931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 641/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Thu Jul 21 22:23:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:33.715654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4542d-8994-4e1e-9b08-8422e352d4e5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:33.806915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:33.817029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4542d-8994-4e1e-9b08-8422e352d4e5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:33.856436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f31900-02d5-4bbf-801f-b1ee9ef34446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:33.857867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f31900-02d5-4bbf-801f-b1ee9ef34446 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:33.858253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f31900-02d5-4bbf-801f-b1ee9ef34446 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:33.858438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f31900-02d5-4bbf-801f-b1ee9ef34446 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:33.929779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f31900-02d5-4bbf-801f-b1ee9ef34446 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:33.931244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f31900-02d5-4bbf-801f-b1ee9ef34446 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:34.020464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:34.031508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 641/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Thu Jul 21 22:23:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:23:34.037338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a2d08-c493-4a79-8150-8ea4a3167604 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:34.097776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:34.139453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a2d08-c493-4a79-8150-8ea4a3167604 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:34.173296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928f2fe-9674-4623-b024-01e8ed42bc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:34.174095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928f2fe-9674-4623-b024-01e8ed42bc4f None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:34.174382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928f2fe-9674-4623-b024-01e8ed42bc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:34.174660 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928f2fe-9674-4623-b024-01e8ed42bc4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:34.184594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928f2fe-9674-4623-b024-01e8ed42bc4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:34.186594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928f2fe-9674-4623-b024-01e8ed42bc4f None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-2928f2fe-9674-4623-b024-01e8ed42bc4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:23:34.203914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:34.205031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:34.205982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:23:34.205982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:23:34.205982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:23:34.205982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:34.205982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 642/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:34.210864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f31900-02d5-4bbf-801f-b1ee9ef34446 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:34.320674 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f31900-02d5-4bbf-801f-b1ee9ef34446 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:34.364428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d0f45-2b52-4d79-b0de-f769cbb63949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:34.365063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d0f45-2b52-4d79-b0de-f769cbb63949 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:34.365349 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d0f45-2b52-4d79-b0de-f769cbb63949 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:34.365630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d0f45-2b52-4d79-b0de-f769cbb63949 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:34.366374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d0f45-2b52-4d79-b0de-f769cbb63949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:34.366687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d0f45-2b52-4d79-b0de-f769cbb63949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:34.368095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d0f45-2b52-4d79-b0de-f769cbb63949 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:34.380352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 642/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Thu Jul 21 22:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:34.385983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928f2fe-9674-4623-b024-01e8ed42bc4f None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:34.484299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928f2fe-9674-4623-b024-01e8ed42bc4f None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:34.498679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:34.518505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32b289-0a4b-46fc-a21d-91792cde9ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:34.519302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32b289-0a4b-46fc-a21d-91792cde9ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:34.519585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32b289-0a4b-46fc-a21d-91792cde9ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:34.519861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32b289-0a4b-46fc-a21d-91792cde9ff6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:34.526494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32b289-0a4b-46fc-a21d-91792cde9ff6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:34.527919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32b289-0a4b-46fc-a21d-91792cde9ff6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-5c32b289-0a4b-46fc-a21d-91792cde9ff6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:23:34.541711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:34.542807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:34.544003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:23:34.544003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:23:34.544003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:23:34.544003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:23:34.544003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:23:34.544003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:23:34.544003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:23:34.544003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 22:23:34.544003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 643/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Thu Jul 21 22:23:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:23:34.549409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d0f45-2b52-4d79-b0de-f769cbb63949 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:34.599663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:34.662961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d0f45-2b52-4d79-b0de-f769cbb63949 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:34.707747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e5a7b-4b54-44ff-bb2e-46157506a662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:34.708738 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e5a7b-4b54-44ff-bb2e-46157506a662 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:34.709204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e5a7b-4b54-44ff-bb2e-46157506a662 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:34.709645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e5a7b-4b54-44ff-bb2e-46157506a662 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:34.710522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e5a7b-4b54-44ff-bb2e-46157506a662 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:34.710828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e5a7b-4b54-44ff-bb2e-46157506a662 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:34.712331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e5a7b-4b54-44ff-bb2e-46157506a662 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:34.728031 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 643/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Thu Jul 21 22:23:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:34.734757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32b289-0a4b-46fc-a21d-91792cde9ff6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:34.832506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32b289-0a4b-46fc-a21d-91792cde9ff6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:34.865587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5f913-7574-4a5c-aef1-bc323eada191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:34.866237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5f913-7574-4a5c-aef1-bc323eada191 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:34.866518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5f913-7574-4a5c-aef1-bc323eada191 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:34.866790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5f913-7574-4a5c-aef1-bc323eada191 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:34.867617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5f913-7574-4a5c-aef1-bc323eada191 None admin] RBAC: Adding query filter params (role.id=f957e09d047b47cfa0891d25b4181263, scope.project.id=b0638fe2ce7a40299f2d654f58ed535c, user.id=705ff0e73e95499e873b1df2a24fefe8) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:34.867932 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5f913-7574-4a5c-aef1-bc323eada191 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:34.869378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5f913-7574-4a5c-aef1-bc323eada191 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:34.875338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 644/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Thu Jul 21 22:23:34 2022] GET /identity/v3/role_assignments?user.id=705ff0e73e95499e873b1df2a24fefe8&scope.project.id=b0638fe2ce7a40299f2d654f58ed535c&role.id=f957e09d047b47cfa0891d25b4181263 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:37.023290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9963fe-2cec-40d1-9257-cd1d702c72b0 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:37.023848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9963fe-2cec-40d1-9257-cd1d702c72b0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:37.023848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9963fe-2cec-40d1-9257-cd1d702c72b0 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:37.025273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 644/1288] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:37.030882 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bd3e6-852a-40b9-b4b4-7a4def7db1be None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:37.031336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bd3e6-852a-40b9-b4b4-7a4def7db1be None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:37.031448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bd3e6-852a-40b9-b4b4-7a4def7db1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:37.050509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-124bd3e6-852a-40b9-b4b4-7a4def7db1be None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:37.051696 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-124bd3e6-852a-40b9-b4b4-7a4def7db1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:37.081459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:37.119783 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 645/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:37.145747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9963fe-2cec-40d1-9257-cd1d702c72b0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:37.261018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9963fe-2cec-40d1-9257-cd1d702c72b0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:37.273579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:37.299423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9fd4b-35b6-423f-b623-d6eee605fd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:37.300309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9fd4b-35b6-423f-b623-d6eee605fd71 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:37.300736 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9fd4b-35b6-423f-b623-d6eee605fd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:37.301017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9fd4b-35b6-423f-b623-d6eee605fd71 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:37.301772 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9fd4b-35b6-423f-b623-d6eee605fd71 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:37.303448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9fd4b-35b6-423f-b623-d6eee605fd71 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-1be9fd4b-35b6-423f-b623-d6eee605fd71 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:37.315659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:37.317024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:37.318276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 645/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 22:23:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:23:37.323051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bd3e6-852a-40b9-b4b4-7a4def7db1be None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:37.423932 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bd3e6-852a-40b9-b4b4-7a4def7db1be None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:37.460546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4ee6d-3786-4725-9f46-a57aedc86541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:37.461223 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4ee6d-3786-4725-9f46-a57aedc86541 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:37.461510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4ee6d-3786-4725-9f46-a57aedc86541 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:37.461790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4ee6d-3786-4725-9f46-a57aedc86541 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:37.462508 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4ee6d-3786-4725-9f46-a57aedc86541 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:37.462818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4ee6d-3786-4725-9f46-a57aedc86541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:37.464181 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4ee6d-3786-4725-9f46-a57aedc86541 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:37.469671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 646/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Jul 21 22:23:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:37.475527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9fd4b-35b6-423f-b623-d6eee605fd71 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:37.595681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9fd4b-35b6-423f-b623-d6eee605fd71 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:37.629915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49021e83-ba3b-4a86-9905-442cb6297989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:37.630580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49021e83-ba3b-4a86-9905-442cb6297989 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:37.630870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49021e83-ba3b-4a86-9905-442cb6297989 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:37.631149 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49021e83-ba3b-4a86-9905-442cb6297989 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:37.631860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49021e83-ba3b-4a86-9905-442cb6297989 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:37.632169 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49021e83-ba3b-4a86-9905-442cb6297989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:37.633574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49021e83-ba3b-4a86-9905-442cb6297989 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:37.638426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 646/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Jul 21 22:23:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:23:39.685553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e8b45-90d7-4f1e-a3fa-6473762705bd None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:39.686196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e8b45-90d7-4f1e-a3fa-6473762705bd None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:39.686360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e8b45-90d7-4f1e-a3fa-6473762705bd None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:39.687789 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 647/1293] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:23:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:39.693442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51358e3-ff82-4966-b84f-44cceab89a67 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:39.693926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51358e3-ff82-4966-b84f-44cceab89a67 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:39.694022 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51358e3-ff82-4966-b84f-44cceab89a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:39.710913 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c51358e3-ff82-4966-b84f-44cceab89a67 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:39.713098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c51358e3-ff82-4966-b84f-44cceab89a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:39.744194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:39.783267 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 647/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:39.806770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e8b45-90d7-4f1e-a3fa-6473762705bd None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:39.838350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:39.917432 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e8b45-90d7-4f1e-a3fa-6473762705bd None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:39.952708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea99cab-afd5-4743-b86f-f40cc4a46f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:39.953294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea99cab-afd5-4743-b86f-f40cc4a46f95 None admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:39.953577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea99cab-afd5-4743-b86f-f40cc4a46f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:39.953851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea99cab-afd5-4743-b86f-f40cc4a46f95 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:39.954567 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea99cab-afd5-4743-b86f-f40cc4a46f95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:39.955919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea99cab-afd5-4743-b86f-f40cc4a46f95 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:39.970568 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 648/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Thu Jul 21 22:23:39 2022] POST /identity/v3/services => generated 246 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:23:42.053105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b62eaa-6031-40d6-9b29-5c028705cffa None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:42.053647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b62eaa-6031-40d6-9b29-5c028705cffa None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:42.053961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b62eaa-6031-40d6-9b29-5c028705cffa None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:42.055561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 648/1296] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:23:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:42.062691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e719d7e-179e-49c7-a4ff-9caf4df42267 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:42.063213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e719d7e-179e-49c7-a4ff-9caf4df42267 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:42.063414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e719d7e-179e-49c7-a4ff-9caf4df42267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:42.081143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-8e719d7e-179e-49c7-a4ff-9caf4df42267 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:42.082353 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8e719d7e-179e-49c7-a4ff-9caf4df42267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:42.165189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 649/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:42 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:42.190475 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b62eaa-6031-40d6-9b29-5c028705cffa None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:42.305914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b62eaa-6031-40d6-9b29-5c028705cffa None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:42.341622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2309f8-b2f7-4fc1-b9e3-ab7a1838e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:42.342418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2309f8-b2f7-4fc1-b9e3-ab7a1838e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:42.342730 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2309f8-b2f7-4fc1-b9e3-ab7a1838e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:42.343011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2309f8-b2f7-4fc1-b9e3-ab7a1838e9e4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:42.343673 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2309f8-b2f7-4fc1-b9e3-ab7a1838e9e4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:42.345096 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2309f8-b2f7-4fc1-b9e3-ab7a1838e9e4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-1f2309f8-b2f7-4fc1-b9e3-ab7a1838e9e4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:42.359116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:42.360243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:42.361246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 649/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 22:23:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:42.366680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e719d7e-179e-49c7-a4ff-9caf4df42267 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:42.385922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:42.474448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e719d7e-179e-49c7-a4ff-9caf4df42267 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:42.509764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71447f2a-de90-49fc-ad00-81921e42a1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:42.510431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71447f2a-de90-49fc-ad00-81921e42a1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:42.510710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71447f2a-de90-49fc-ad00-81921e42a1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:42.510988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71447f2a-de90-49fc-ad00-81921e42a1c6 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:42.511705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71447f2a-de90-49fc-ad00-81921e42a1c6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:42.512011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71447f2a-de90-49fc-ad00-81921e42a1c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:42.514055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71447f2a-de90-49fc-ad00-81921e42a1c6 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:42.519381 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 650/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Jul 21 22:23:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:42.526714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2309f8-b2f7-4fc1-b9e3-ab7a1838e9e4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:42.559629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:42.636710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2309f8-b2f7-4fc1-b9e3-ab7a1838e9e4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:42.673357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccc440-1495-4a86-92df-e0f80c919783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:42.674026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccc440-1495-4a86-92df-e0f80c919783 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:42.674316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccc440-1495-4a86-92df-e0f80c919783 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:42.674594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccc440-1495-4a86-92df-e0f80c919783 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:42.675399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccc440-1495-4a86-92df-e0f80c919783 None admin] RBAC: Adding query filter params (interface=public, service_id=874b34278bd1440b86a834d56d514212, region_id=RegionOne) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:42.675712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccc440-1495-4a86-92df-e0f80c919783 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:42.677142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccc440-1495-4a86-92df-e0f80c919783 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:42.682736 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 650/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Thu Jul 21 22:23:42 2022] GET /identity/v3/endpoints?service_id=874b34278bd1440b86a834d56d514212&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:42.688607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71447f2a-de90-49fc-ad00-81921e42a1c6 None admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:42.775745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:42.804196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71447f2a-de90-49fc-ad00-81921e42a1c6 None admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:42.839543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58493e30-c3ca-41ae-a83b-47715d3f862a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:42.840132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58493e30-c3ca-41ae-a83b-47715d3f862a None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:42.840416 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58493e30-c3ca-41ae-a83b-47715d3f862a None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:42.841314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58493e30-c3ca-41ae-a83b-47715d3f862a None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:42.842276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58493e30-c3ca-41ae-a83b-47715d3f862a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:42.843665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58493e30-c3ca-41ae-a83b-47715d3f862a None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:42.851338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 651/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Thu Jul 21 22:23:42 2022] GET /identity/v3/services => generated 2191 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:23:44.905304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a600ca2-f1b2-4c80-95b7-c4fab6d3685a None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:44.905843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a600ca2-f1b2-4c80-95b7-c4fab6d3685a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:44.905843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a600ca2-f1b2-4c80-95b7-c4fab6d3685a None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:44.907363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 651/1302] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:23:44.912681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9e92e-127f-452d-be9d-2058ba5afb08 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:44.913169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9e92e-127f-452d-be9d-2058ba5afb08 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:44.913284 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9e92e-127f-452d-be9d-2058ba5afb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:44.930430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-1cf9e92e-127f-452d-be9d-2058ba5afb08 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:23:44.931622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1cf9e92e-127f-452d-be9d-2058ba5afb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:23:45.000037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 652/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:23:44 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:23:45.023742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a600ca2-f1b2-4c80-95b7-c4fab6d3685a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:45.132432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:45.140973 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a600ca2-f1b2-4c80-95b7-c4fab6d3685a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:45.178814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04560f1d-8d38-4d09-8bc0-219e1d785635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:45.179522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04560f1d-8d38-4d09-8bc0-219e1d785635 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:45.179801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04560f1d-8d38-4d09-8bc0-219e1d785635 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:45.180072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04560f1d-8d38-4d09-8bc0-219e1d785635 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:45.180778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04560f1d-8d38-4d09-8bc0-219e1d785635 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:45.182158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04560f1d-8d38-4d09-8bc0-219e1d785635 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-04560f1d-8d38-4d09-8bc0-219e1d785635 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:23:45.194882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:23:45.195937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:23:45.197193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 652/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Thu Jul 21 22:23:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:23:45.202196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9e92e-127f-452d-be9d-2058ba5afb08 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:45.280922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:45.303355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9e92e-127f-452d-be9d-2058ba5afb08 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:45.336170 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce4731-d892-4790-89b5-04f930748b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:45.336857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce4731-d892-4790-89b5-04f930748b92 None admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:45.337141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce4731-d892-4790-89b5-04f930748b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:45.337417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce4731-d892-4790-89b5-04f930748b92 None admin] PATH_INFO: `/v3/services` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:45.338124 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce4731-d892-4790-89b5-04f930748b92 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:23:45.338431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce4731-d892-4790-89b5-04f930748b92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:45.339790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce4731-d892-4790-89b5-04f930748b92 None admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:45.345067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 653/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Thu Jul 21 22:23:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:23:45.350859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04560f1d-8d38-4d09-8bc0-219e1d785635 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:23:45.453416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04560f1d-8d38-4d09-8bc0-219e1d785635 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:23:45.486387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436d72b-633d-476c-80c3-351f4c0d9bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:23:45.487040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436d72b-633d-476c-80c3-351f4c0d9bfb None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:23:45.487321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436d72b-633d-476c-80c3-351f4c0d9bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:23:45.487595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436d72b-633d-476c-80c3-351f4c0d9bfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:23:45.488346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436d72b-633d-476c-80c3-351f4c0d9bfb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:23:45.489751 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436d72b-633d-476c-80c3-351f4c0d9bfb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:23:45.513417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 653/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Thu Jul 21 22:23:45 2022] POST /identity/v3/endpoints => generated 331 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:23:55.342036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=30 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:23:55.508314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:52.169252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ba1bb-922e-40fb-b894-dfcf6574ab25 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:52.169844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ba1bb-922e-40fb-b894-dfcf6574ab25 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:52.169844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ba1bb-922e-40fb-b894-dfcf6574ab25 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:52.171287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 654/1307] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:24:52.177259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3088f-bb23-4cac-a102-57868334a048 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:52.177823 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3088f-bb23-4cac-a102-57868334a048 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:52.177823 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3088f-bb23-4cac-a102-57868334a048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:52.290534 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-afc3088f-bb23-4cac-a102-57868334a048 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:52.294087 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-afc3088f-bb23-4cac-a102-57868334a048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:52.500870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 654/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:52.583331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167e9fa-d513-4e05-99fa-c3b048ac3159 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:52.583635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167e9fa-d513-4e05-99fa-c3b048ac3159 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:52.583928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167e9fa-d513-4e05-99fa-c3b048ac3159 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:52.585494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 655/1309] 10.222.0.42 () {54 vars in 965 bytes} [Thu Jul 21 22:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:52.600416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98560-c5f9-492f-af1f-8b898d8fde68 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:52.600714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98560-c5f9-492f-af1f-8b898d8fde68 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:52.601077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98560-c5f9-492f-af1f-8b898d8fde68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:52.811772 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cfc98560-c5f9-492f-af1f-8b898d8fde68 None None] MFA Rules not processed for user `8a120d9e7e204dc5b972fd9402897b41`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:52.814162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cfc98560-c5f9-492f-af1f-8b898d8fde68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:52.861544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:52.978565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 655/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Thu Jul 21 22:24:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:52.987588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167e9fa-d513-4e05-99fa-c3b048ac3159 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:52.989804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5167e9fa-d513-4e05-99fa-c3b048ac3159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:53.168585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167e9fa-d513-4e05-99fa-c3b048ac3159 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:53.331979 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013ea03-6988-4068-b61f-26c7cb9179b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:53.333265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013ea03-6988-4068-b61f-26c7cb9179b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:53.333778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013ea03-6988-4068-b61f-26c7cb9179b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:53.334285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013ea03-6988-4068-b61f-26c7cb9179b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:53.338662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0013ea03-6988-4068-b61f-26c7cb9179b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:53.454672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:53.533487 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013ea03-6988-4068-b61f-26c7cb9179b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:53.535552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013ea03-6988-4068-b61f-26c7cb9179b6 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:53.560351 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 656/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:24:56.052868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceed182-bc49-4cc1-beeb-ffe8ac2c93b9 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:56.053489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceed182-bc49-4cc1-beeb-ffe8ac2c93b9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:56.053489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceed182-bc49-4cc1-beeb-ffe8ac2c93b9 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:56.054975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 656/1312] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:56.060942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a644c-1cde-4002-95cc-3b42081fb112 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:56.061499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a644c-1cde-4002-95cc-3b42081fb112 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:56.061499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a644c-1cde-4002-95cc-3b42081fb112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:56.153581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f11a644c-1cde-4002-95cc-3b42081fb112 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:56.154828 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f11a644c-1cde-4002-95cc-3b42081fb112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:56.271363 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 657/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:56.311252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceed182-bc49-4cc1-beeb-ffe8ac2c93b9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:56.312880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cceed182-bc49-4cc1-beeb-ffe8ac2c93b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:56.449140 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceed182-bc49-4cc1-beeb-ffe8ac2c93b9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:56.524225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:56.527250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be740d-cf18-44ab-9037-f1aacaa736bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:56.528144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be740d-cf18-44ab-9037-f1aacaa736bb service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:56.528675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be740d-cf18-44ab-9037-f1aacaa736bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:56.529165 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be740d-cf18-44ab-9037-f1aacaa736bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:56.530921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-87be740d-cf18-44ab-9037-f1aacaa736bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:56.643516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be740d-cf18-44ab-9037-f1aacaa736bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:56.644937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be740d-cf18-44ab-9037-f1aacaa736bb service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:56.664436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 657/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:24:59.174095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7acb7a-89cb-4544-9bee-c132c748ff01 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:59.174706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7acb7a-89cb-4544-9bee-c132c748ff01 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:59.174706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7acb7a-89cb-4544-9bee-c132c748ff01 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:59.176050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 658/1315] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:24:59.182126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9299ef3-af15-4cef-9a0c-09e60d0005cc None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:59.182597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9299ef3-af15-4cef-9a0c-09e60d0005cc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:59.182702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9299ef3-af15-4cef-9a0c-09e60d0005cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:59.255808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f9299ef3-af15-4cef-9a0c-09e60d0005cc None None] MFA Rules not processed for user `67aea1a002a149b2aaa38b0dd4193f16`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:24:59.257075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f9299ef3-af15-4cef-9a0c-09e60d0005cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:59.314090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:59.362227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 658/1316] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:24:59 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:24:59.403935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7acb7a-89cb-4544-9bee-c132c748ff01 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:24:59.405521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6c7acb7a-89cb-4544-9bee-c132c748ff01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:59.415655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:59.514772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7acb7a-89cb-4544-9bee-c132c748ff01 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:24:59.592398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a4a35-367e-4c04-9711-6bb8b4633246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:24:59.593087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a4a35-367e-4c04-9711-6bb8b4633246 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:24:59.593378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a4a35-367e-4c04-9711-6bb8b4633246 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:24:59.593646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a4a35-367e-4c04-9711-6bb8b4633246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:24:59.595383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-327a4a35-367e-4c04-9711-6bb8b4633246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:24:59.667623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:24:59.723173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a4a35-367e-4c04-9711-6bb8b4633246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:24:59.724638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a4a35-367e-4c04-9711-6bb8b4633246 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:24:59.747888 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 659/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:24:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:02.471698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32947ce-49cc-437d-bd1d-2d4b2ef3ea1f None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:02.472297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32947ce-49cc-437d-bd1d-2d4b2ef3ea1f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:02.472297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32947ce-49cc-437d-bd1d-2d4b2ef3ea1f None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:02.473761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 659/1318] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:25:02.479737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6a024-fc63-4e8a-b3e8-2e6826ea6208 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:02.480295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6a024-fc63-4e8a-b3e8-2e6826ea6208 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:02.480295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6a024-fc63-4e8a-b3e8-2e6826ea6208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:02.556305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-73c6a024-fc63-4e8a-b3e8-2e6826ea6208 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:02.557583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-73c6a024-fc63-4e8a-b3e8-2e6826ea6208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:02.665370 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 660/1319] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:02.708438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32947ce-49cc-437d-bd1d-2d4b2ef3ea1f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:02.708438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b32947ce-49cc-437d-bd1d-2d4b2ef3ea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:02.816766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32947ce-49cc-437d-bd1d-2d4b2ef3ea1f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:02.902009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29951027-81f1-41db-adf6-5763b058d3d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:02.902669 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29951027-81f1-41db-adf6-5763b058d3d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:02.902956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29951027-81f1-41db-adf6-5763b058d3d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:02.903236 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29951027-81f1-41db-adf6-5763b058d3d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:02.905144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-29951027-81f1-41db-adf6-5763b058d3d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:02.914886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:03.032789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29951027-81f1-41db-adf6-5763b058d3d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:03.034151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29951027-81f1-41db-adf6-5763b058d3d4 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:03.052976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 660/1320] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:25:05.969131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54312cb5-9a0b-4a80-8671-e6284a561f8a None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:05.970016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54312cb5-9a0b-4a80-8671-e6284a561f8a None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:05.970180 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54312cb5-9a0b-4a80-8671-e6284a561f8a None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:05.971592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 661/1321] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:05.977500 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad173f-9da3-4d7e-90da-234b71359373 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:05.978005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad173f-9da3-4d7e-90da-234b71359373 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:05.978213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad173f-9da3-4d7e-90da-234b71359373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:06.045740 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7fad173f-9da3-4d7e-90da-234b71359373 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:06.046998 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7fad173f-9da3-4d7e-90da-234b71359373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:06.115840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:06.182471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 661/1322] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:06.230603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54312cb5-9a0b-4a80-8671-e6284a561f8a None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:06.232165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-54312cb5-9a0b-4a80-8671-e6284a561f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:06.253409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:06.345425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54312cb5-9a0b-4a80-8671-e6284a561f8a None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:06.422267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee6f79-5f2e-4ec1-a3e3-c9c8a1965d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:06.422916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee6f79-5f2e-4ec1-a3e3-c9c8a1965d17 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:06.423199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee6f79-5f2e-4ec1-a3e3-c9c8a1965d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:06.423473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee6f79-5f2e-4ec1-a3e3-c9c8a1965d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:06.425179 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-10ee6f79-5f2e-4ec1-a3e3-c9c8a1965d17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:06.502916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:06.556026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee6f79-5f2e-4ec1-a3e3-c9c8a1965d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:06.557551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee6f79-5f2e-4ec1-a3e3-c9c8a1965d17 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:06.581371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 662/1323] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:08.837689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52850c18-ffc3-4322-a90d-448645147c88 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:08.838263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52850c18-ffc3-4322-a90d-448645147c88 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:08.838363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52850c18-ffc3-4322-a90d-448645147c88 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:08.839841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 662/1324] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:08.845951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a18b04-302f-46a3-b033-9532818f8418 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:08.846767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a18b04-302f-46a3-b033-9532818f8418 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:08.846767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a18b04-302f-46a3-b033-9532818f8418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:08.921884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-a6a18b04-302f-46a3-b033-9532818f8418 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:08.923159 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a6a18b04-302f-46a3-b033-9532818f8418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:09.037715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 663/1325] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:09.090742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82205865-2e4b-4bb9-a80c-5522fc07d3f1 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:09.091032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82205865-2e4b-4bb9-a80c-5522fc07d3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:09.091306 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82205865-2e4b-4bb9-a80c-5522fc07d3f1 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:09.092746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 663/1326] 10.222.0.42 () {54 vars in 965 bytes} [Thu Jul 21 22:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:09.099118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cf9ba-3ae0-42f0-92bc-29b0a663f61a None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:09.099506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cf9ba-3ae0-42f0-92bc-29b0a663f61a None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:09.099862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cf9ba-3ae0-42f0-92bc-29b0a663f61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:09.137902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:09.193847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-1e7cf9ba-3ae0-42f0-92bc-29b0a663f61a None None] MFA Rules not processed for user `8a120d9e7e204dc5b972fd9402897b41`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:09.195671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1e7cf9ba-3ae0-42f0-92bc-29b0a663f61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:09.285539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 664/1327] 10.222.0.42 () {58 vars in 1062 bytes} [Thu Jul 21 22:25:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:09.295775 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82205865-2e4b-4bb9-a80c-5522fc07d3f1 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:09.297313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-82205865-2e4b-4bb9-a80c-5522fc07d3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:09.425064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82205865-2e4b-4bb9-a80c-5522fc07d3f1 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:09.522293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf4021-4c93-4a60-827f-e014e9c8e132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:09.523459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf4021-4c93-4a60-827f-e014e9c8e132 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:09.523933 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf4021-4c93-4a60-827f-e014e9c8e132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:09.524437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf4021-4c93-4a60-827f-e014e9c8e132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:09.527745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bbbf4021-4c93-4a60-827f-e014e9c8e132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:09.559469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:09.728751 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf4021-4c93-4a60-827f-e014e9c8e132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:09.730451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf4021-4c93-4a60-827f-e014e9c8e132 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:09.766456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 664/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:12.228290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783ed9b-46e6-4f86-9968-afd5c45b9cc3 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:12.228905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783ed9b-46e6-4f86-9968-afd5c45b9cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:12.229263 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783ed9b-46e6-4f86-9968-afd5c45b9cc3 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:12.231665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 665/1329] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:25:12.239538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88061e1-da6d-4f01-9cb1-5c2933d9c013 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:12.240331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88061e1-da6d-4f01-9cb1-5c2933d9c013 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:12.240555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88061e1-da6d-4f01-9cb1-5c2933d9c013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:12.331418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f88061e1-da6d-4f01-9cb1-5c2933d9c013 None None] MFA Rules not processed for user `67aea1a002a149b2aaa38b0dd4193f16`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:12.333616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f88061e1-da6d-4f01-9cb1-5c2933d9c013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:12.422592 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:12.474084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 665/1330] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:12.527914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783ed9b-46e6-4f86-9968-afd5c45b9cc3 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:12.529527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1783ed9b-46e6-4f86-9968-afd5c45b9cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:12.652794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783ed9b-46e6-4f86-9968-afd5c45b9cc3 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:12.677104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:12.746202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e5abe-cbc8-4b3a-a200-f07483f36c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:12.746866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e5abe-cbc8-4b3a-a200-f07483f36c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:12.747165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e5abe-cbc8-4b3a-a200-f07483f36c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:12.747445 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e5abe-cbc8-4b3a-a200-f07483f36c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:12.749240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-967e5abe-cbc8-4b3a-a200-f07483f36c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:12.882548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967e5abe-cbc8-4b3a-a200-f07483f36c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:12.884020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967e5abe-cbc8-4b3a-a200-f07483f36c88 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:12.905173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 666/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:12 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:15.511825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea10c1-a6b4-4efd-b1a6-30a213b90ae6 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:15.512346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea10c1-a6b4-4efd-b1a6-30a213b90ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:15.512346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea10c1-a6b4-4efd-b1a6-30a213b90ae6 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:15.513714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 666/1332] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:15.519187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e47261-12d4-4ed9-818f-3f47f710c288 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:15.519604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e47261-12d4-4ed9-818f-3f47f710c288 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:15.519743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e47261-12d4-4ed9-818f-3f47f710c288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:15.582407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:15.585186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-b8e47261-12d4-4ed9-818f-3f47f710c288 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:15.587585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b8e47261-12d4-4ed9-818f-3f47f710c288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:15.687693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 667/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:15.725699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea10c1-a6b4-4efd-b1a6-30a213b90ae6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:15.728847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e6ea10c1-a6b4-4efd-b1a6-30a213b90ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:15.839824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea10c1-a6b4-4efd-b1a6-30a213b90ae6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:15.914417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18299ac5-73f0-480d-a0fc-d2bc35550a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:15.915091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18299ac5-73f0-480d-a0fc-d2bc35550a66 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:15.915361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18299ac5-73f0-480d-a0fc-d2bc35550a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:15.915639 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18299ac5-73f0-480d-a0fc-d2bc35550a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:15.917359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-18299ac5-73f0-480d-a0fc-d2bc35550a66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:15.945739 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:16.029858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18299ac5-73f0-480d-a0fc-d2bc35550a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:16.031322 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18299ac5-73f0-480d-a0fc-d2bc35550a66 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:16.049299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 667/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:18.279557 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54bf39-73b1-4a63-9167-dc6b0590f406 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:18.280122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54bf39-73b1-4a63-9167-dc6b0590f406 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:18.280122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54bf39-73b1-4a63-9167-dc6b0590f406 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:18.281635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 668/1335] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:18.287231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa9a6d-0322-4de3-830e-83a6d2453cee None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:18.287683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa9a6d-0322-4de3-830e-83a6d2453cee None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:18.287776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa9a6d-0322-4de3-830e-83a6d2453cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:18.363458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-82aa9a6d-0322-4de3-830e-83a6d2453cee None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:18.364789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-82aa9a6d-0322-4de3-830e-83a6d2453cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:18.448439 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:18.475349 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 668/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:18.519251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54bf39-73b1-4a63-9167-dc6b0590f406 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:18.520880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fe54bf39-73b1-4a63-9167-dc6b0590f406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:18.648561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54bf39-73b1-4a63-9167-dc6b0590f406 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:18.681758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:18.730233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f3555-2bc6-47f9-adb0-9a057c9a61b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:18.730892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3555-2bc6-47f9-adb0-9a057c9a61b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:18.731177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3555-2bc6-47f9-adb0-9a057c9a61b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:18.731454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3555-2bc6-47f9-adb0-9a057c9a61b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:18.733206 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8b9f3555-2bc6-47f9-adb0-9a057c9a61b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:18.889489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f3555-2bc6-47f9-adb0-9a057c9a61b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:18.891040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f3555-2bc6-47f9-adb0-9a057c9a61b4 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:18.912006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 669/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:21.451577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908693b-5cc7-4d34-83e1-bf41a981a1a5 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:21.452181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908693b-5cc7-4d34-83e1-bf41a981a1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:21.452181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908693b-5cc7-4d34-83e1-bf41a981a1a5 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:21.453590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 669/1338] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:25:21.463444 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491c2f2-013b-4ff8-a264-e19744bf46bc None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:21.464062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491c2f2-013b-4ff8-a264-e19744bf46bc None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:21.464317 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491c2f2-013b-4ff8-a264-e19744bf46bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:21.533478 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7491c2f2-013b-4ff8-a264-e19744bf46bc None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:21.534698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7491c2f2-013b-4ff8-a264-e19744bf46bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:21.544184 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:21.637217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 670/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:21.673264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908693b-5cc7-4d34-83e1-bf41a981a1a5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:21.674813 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e908693b-5cc7-4d34-83e1-bf41a981a1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:21.788929 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908693b-5cc7-4d34-83e1-bf41a981a1a5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:21.870665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7e4b5-c8a6-45af-8ac0-2efcf1a84644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:21.871323 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7e4b5-c8a6-45af-8ac0-2efcf1a84644 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:21.871606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7e4b5-c8a6-45af-8ac0-2efcf1a84644 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:21.871883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7e4b5-c8a6-45af-8ac0-2efcf1a84644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:21.873646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-30f7e4b5-c8a6-45af-8ac0-2efcf1a84644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:21.913174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:22.024184 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7e4b5-c8a6-45af-8ac0-2efcf1a84644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:22.025668 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7e4b5-c8a6-45af-8ac0-2efcf1a84644 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:22.045495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 670/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:26.432720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f8093-0cfc-41b5-9fe1-91e0e4e65ad1 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:26.433605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f8093-0cfc-41b5-9fe1-91e0e4e65ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:26.433605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f8093-0cfc-41b5-9fe1-91e0e4e65ad1 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:26.434986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 671/1341] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:26.441808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6818f1-49f9-46d0-8fdf-de990ef32a57 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:26.442404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6818f1-49f9-46d0-8fdf-de990ef32a57 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:26.443015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6818f1-49f9-46d0-8fdf-de990ef32a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:26.532515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ed6818f1-49f9-46d0-8fdf-de990ef32a57 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:26.534106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ed6818f1-49f9-46d0-8fdf-de990ef32a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:26.651154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:26.665004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 671/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:26.706258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f8093-0cfc-41b5-9fe1-91e0e4e65ad1 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:26.709314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-043f8093-0cfc-41b5-9fe1-91e0e4e65ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:26.876366 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f8093-0cfc-41b5-9fe1-91e0e4e65ad1 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:26.946922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:26.989273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597571e6-018c-4dfd-8e0f-2abe08c38f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:26.989937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597571e6-018c-4dfd-8e0f-2abe08c38f68 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:26.990226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597571e6-018c-4dfd-8e0f-2abe08c38f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:26.990507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597571e6-018c-4dfd-8e0f-2abe08c38f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:26.992772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-597571e6-018c-4dfd-8e0f-2abe08c38f68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:27.118721 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597571e6-018c-4dfd-8e0f-2abe08c38f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:27.120181 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597571e6-018c-4dfd-8e0f-2abe08c38f68 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:27.143784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 672/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:30.410421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26212e-cada-4046-9340-c89ebb1855cd None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:30.411598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26212e-cada-4046-9340-c89ebb1855cd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:30.411598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26212e-cada-4046-9340-c89ebb1855cd None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:30.412404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 672/1344] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:25:30.419186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033333a-5471-414e-81b4-b8ad52348764 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:30.419970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033333a-5471-414e-81b4-b8ad52348764 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:30.419970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033333a-5471-414e-81b4-b8ad52348764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:30.503433 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7033333a-5471-414e-81b4-b8ad52348764 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:30.504685 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7033333a-5471-414e-81b4-b8ad52348764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:30.524576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:30.621691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 673/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:30.663814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26212e-cada-4046-9340-c89ebb1855cd None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:30.665626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8d26212e-cada-4046-9340-c89ebb1855cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:30.807963 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26212e-cada-4046-9340-c89ebb1855cd None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:30.889940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5978e1-1533-415b-b2d2-5aaac3787551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:30.890601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5978e1-1533-415b-b2d2-5aaac3787551 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:30.890886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5978e1-1533-415b-b2d2-5aaac3787551 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:30.891162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5978e1-1533-415b-b2d2-5aaac3787551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:30.892901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2a5978e1-1533-415b-b2d2-5aaac3787551 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:30.941417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:31.018842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5978e1-1533-415b-b2d2-5aaac3787551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:31.020310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5978e1-1533-415b-b2d2-5aaac3787551 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:31.040565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 673/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:35.501177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2f75b-930e-4a1b-9593-70db44751b34 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:35.501926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2f75b-930e-4a1b-9593-70db44751b34 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:35.501926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2f75b-930e-4a1b-9593-70db44751b34 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:35.503114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 674/1347] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:35.511084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a062008-4e79-4929-a497-1703efa44973 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:35.511658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a062008-4e79-4929-a497-1703efa44973 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:35.511795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a062008-4e79-4929-a497-1703efa44973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:35.596714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2a062008-4e79-4929-a497-1703efa44973 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:35.597967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2a062008-4e79-4929-a497-1703efa44973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:35.725230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:35.730048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 674/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:35.771910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2f75b-930e-4a1b-9593-70db44751b34 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:35.774379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-73d2f75b-930e-4a1b-9593-70db44751b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:35.920131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2f75b-930e-4a1b-9593-70db44751b34 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:35.980458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:36.017043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac390998-1ace-4ed0-a3e9-445e58987dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:36.017738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac390998-1ace-4ed0-a3e9-445e58987dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:36.018024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac390998-1ace-4ed0-a3e9-445e58987dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:36.018304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac390998-1ace-4ed0-a3e9-445e58987dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:36.020052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ac390998-1ace-4ed0-a3e9-445e58987dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:36.210121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac390998-1ace-4ed0-a3e9-445e58987dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:36.213082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac390998-1ace-4ed0-a3e9-445e58987dd1 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:36.247309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 675/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:41.606584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26748b-db37-491b-adcb-8eb1bff504e7 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:41.606584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26748b-db37-491b-adcb-8eb1bff504e7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:41.606584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26748b-db37-491b-adcb-8eb1bff504e7 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:41.607750 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 675/1350] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 22:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:41.613452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5aab4-e210-4273-a45e-c53437286958 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:41.614312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5aab4-e210-4273-a45e-c53437286958 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:41.614814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5aab4-e210-4273-a45e-c53437286958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:41.778182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-16f5aab4-e210-4273-a45e-c53437286958 None None] MFA Rules not processed for user `705ff0e73e95499e873b1df2a24fefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:41.779912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-16f5aab4-e210-4273-a45e-c53437286958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:41.803385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:41.916255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 676/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 22:25:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:46.392670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd25383-91b8-4a42-b672-10178f408153 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:46.393543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd25383-91b8-4a42-b672-10178f408153 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:46.393824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd25383-91b8-4a42-b672-10178f408153 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:46.395798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 676/1352] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 22:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:46.403101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1c7e3-8185-4bec-ac18-0afe3dda2258 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:46.403661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1c7e3-8185-4bec-ac18-0afe3dda2258 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:46.403938 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1c7e3-8185-4bec-ac18-0afe3dda2258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:46.508857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-aca1c7e3-8185-4bec-ac18-0afe3dda2258 None None] MFA Rules not processed for user `705ff0e73e95499e873b1df2a24fefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:46.510089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-aca1c7e3-8185-4bec-ac18-0afe3dda2258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:46.601627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:46.651886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 677/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 22:25:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:48.077005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ffeed-46ac-4179-9ef8-6cb3a53d6087 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:48.077617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ffeed-46ac-4179-9ef8-6cb3a53d6087 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:48.077617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ffeed-46ac-4179-9ef8-6cb3a53d6087 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:48.078955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 677/1354] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 22:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:25:48.086507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbc01b-0879-47e3-b404-4c4b584fe301 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:48.087057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbc01b-0879-47e3-b404-4c4b584fe301 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:48.087671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbc01b-0879-47e3-b404-4c4b584fe301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:48.192357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-dadbc01b-0879-47e3-b404-4c4b584fe301 None None] MFA Rules not processed for user `705ff0e73e95499e873b1df2a24fefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:48.193606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-dadbc01b-0879-47e3-b404-4c4b584fe301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:48.322003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 678/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 22:25:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:25:51.956164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac3b9-3c05-49a3-97fb-4c615b93dee5 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:51.957018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac3b9-3c05-49a3-97fb-4c615b93dee5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:51.957018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac3b9-3c05-49a3-97fb-4c615b93dee5 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:51.958153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 678/1356] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:51.967084 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62e104-7883-4735-bc3d-c05e10f5bd96 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:51.967566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62e104-7883-4735-bc3d-c05e10f5bd96 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:51.967726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62e104-7883-4735-bc3d-c05e10f5bd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:52.037393 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:52.084890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-cc62e104-7883-4735-bc3d-c05e10f5bd96 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:52.086107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-cc62e104-7883-4735-bc3d-c05e10f5bd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:52.293461 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 679/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:52.383268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737536f-4f27-4eee-ac6f-3b44babd1d7f None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:52.384724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737536f-4f27-4eee-ac6f-3b44babd1d7f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:52.385348 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737536f-4f27-4eee-ac6f-3b44babd1d7f None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:52.389898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 679/1358] 10.222.0.42 () {54 vars in 962 bytes} [Thu Jul 21 22:25:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:52.399057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4059e-c861-4142-a4b8-11cb4f4d89eb None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:52.400455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4059e-c861-4142-a4b8-11cb4f4d89eb None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:52.401150 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4059e-c861-4142-a4b8-11cb4f4d89eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:52.545456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7fb4059e-c861-4142-a4b8-11cb4f4d89eb None None] MFA Rules not processed for user `c77f6ccc7cb84c04ac532007a04a0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:52.547676 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7fb4059e-c861-4142-a4b8-11cb4f4d89eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:52.641983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:52.761881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 680/1359] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Jul 21 22:25:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:52.769685 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737536f-4f27-4eee-ac6f-3b44babd1d7f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:52.771462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0737536f-4f27-4eee-ac6f-3b44babd1d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:52.985881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737536f-4f27-4eee-ac6f-3b44babd1d7f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:53.079105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:53.130958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc9002-ac26-46ab-9b1e-239fb78c7ca2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:53.131836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc9002-ac26-46ab-9b1e-239fb78c7ca2 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:53.132142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc9002-ac26-46ab-9b1e-239fb78c7ca2 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:53.133157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc9002-ac26-46ab-9b1e-239fb78c7ca2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:53.135618 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c4fc9002-ac26-46ab-9b1e-239fb78c7ca2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:53.250834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc9002-ac26-46ab-9b1e-239fb78c7ca2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:53.252344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc9002-ac26-46ab-9b1e-239fb78c7ca2 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:53.277500 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 680/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:25:55.874038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062610c7-5a42-4bde-8f1f-51c96c1725ed None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:55.874792 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062610c7-5a42-4bde-8f1f-51c96c1725ed None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:55.874792 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062610c7-5a42-4bde-8f1f-51c96c1725ed None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:55.875965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 681/1361] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:55.886731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2176ca-cf56-4265-9bd7-b33916aba473 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:55.887290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2176ca-cf56-4265-9bd7-b33916aba473 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:55.887391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2176ca-cf56-4265-9bd7-b33916aba473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:55.970839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0b2176ca-cf56-4265-9bd7-b33916aba473 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:55.972103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0b2176ca-cf56-4265-9bd7-b33916aba473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:56.019961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:56.100692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 681/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:56.143673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062610c7-5a42-4bde-8f1f-51c96c1725ed None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:56.145378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-062610c7-5a42-4bde-8f1f-51c96c1725ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:56.324540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062610c7-5a42-4bde-8f1f-51c96c1725ed None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:25:56.334733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:56.490605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c5058-7999-4b65-9d41-a653f6fb8be4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:25:56.491768 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c5058-7999-4b65-9d41-a653f6fb8be4 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:56.491768 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c5058-7999-4b65-9d41-a653f6fb8be4 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:56.492325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c5058-7999-4b65-9d41-a653f6fb8be4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:56.494156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-656c5058-7999-4b65-9d41-a653f6fb8be4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:56.610543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c5058-7999-4b65-9d41-a653f6fb8be4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:25:56.612064 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c5058-7999-4b65-9d41-a653f6fb8be4 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:25:56.640596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 682/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:25:59.267781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a69425-bcf9-4099-a86a-a51ada8cd73b None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:59.268735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a69425-bcf9-4099-a86a-a51ada8cd73b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:59.268735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a69425-bcf9-4099-a86a-a51ada8cd73b None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:59.270978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 682/1364] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:25:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:59.277950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f928e-18d2-4ae9-8260-67e79f192dfb None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:59.278925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f928e-18d2-4ae9-8260-67e79f192dfb None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:59.279382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f928e-18d2-4ae9-8260-67e79f192dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:59.306634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:59.380156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-4a3f928e-18d2-4ae9-8260-67e79f192dfb None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:59.381929 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4a3f928e-18d2-4ae9-8260-67e79f192dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:59.534014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 683/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:25:59.585367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c0e3-1803-4dc2-88e5-54dd31486b3d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:59.585913 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c0e3-1803-4dc2-88e5-54dd31486b3d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:59.586436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29c0e3-1803-4dc2-88e5-54dd31486b3d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:59.589305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 683/1366] 10.222.0.42 () {54 vars in 962 bytes} [Thu Jul 21 22:25:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:25:59.599489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f35fb4-2e34-451b-b5c5-e54242b02731 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:25:59.600189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f35fb4-2e34-451b-b5c5-e54242b02731 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:25:59.600701 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f35fb4-2e34-451b-b5c5-e54242b02731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:25:59.707710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-72f35fb4-2e34-451b-b5c5-e54242b02731 None None] MFA Rules not processed for user `c77f6ccc7cb84c04ac532007a04a0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:25:59.709221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-72f35fb4-2e34-451b-b5c5-e54242b02731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:25:59.758351 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:25:59.897123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 684/1367] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Jul 21 22:25:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:25:59.905682 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29c0e3-1803-4dc2-88e5-54dd31486b3d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:25:59.907355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fb29c0e3-1803-4dc2-88e5-54dd31486b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:00.170635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29c0e3-1803-4dc2-88e5-54dd31486b3d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:00.200619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:00.341656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b4aed-2319-44b2-9391-fb0e895f20ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:00.342919 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b4aed-2319-44b2-9391-fb0e895f20ea service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:00.343272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b4aed-2319-44b2-9391-fb0e895f20ea service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:00.343730 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b4aed-2319-44b2-9391-fb0e895f20ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:00.347047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-029b4aed-2319-44b2-9391-fb0e895f20ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:00.475459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b4aed-2319-44b2-9391-fb0e895f20ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:00.477342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b4aed-2319-44b2-9391-fb0e895f20ea service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:00.497848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 684/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:25:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:26:01.453444 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51385a7b-8b5f-46c2-b098-948504f946bc None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:01.453972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51385a7b-8b5f-46c2-b098-948504f946bc None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:01.454205 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51385a7b-8b5f-46c2-b098-948504f946bc None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:01.456251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 685/1369] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:01.462054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc95325-1ce6-4669-b39c-2e7bc221ba09 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:01.462955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc95325-1ce6-4669-b39c-2e7bc221ba09 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:01.463444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc95325-1ce6-4669-b39c-2e7bc221ba09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:01.498741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:01.540241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8fc95325-1ce6-4669-b39c-2e7bc221ba09 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:01.541192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8fc95325-1ce6-4669-b39c-2e7bc221ba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:01.655949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 685/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:01.979057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10813dce-c6d4-43c6-a471-59e71a1ee368 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:01.979504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10813dce-c6d4-43c6-a471-59e71a1ee368 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:01.979671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10813dce-c6d4-43c6-a471-59e71a1ee368 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:01.981069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 686/1371] 10.222.0.42 () {54 vars in 964 bytes} [Thu Jul 21 22:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:01.986573 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda75d5-295d-4266-b4a7-5aea0671c95d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:01.986940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda75d5-295d-4266-b4a7-5aea0671c95d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:01.987221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda75d5-295d-4266-b4a7-5aea0671c95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:02.086927 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-efda75d5-295d-4266-b4a7-5aea0671c95d None None] MFA Rules not processed for user `a697d1e7e42048a8a1fb997bcc7b3fab`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:02.086927 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-efda75d5-295d-4266-b4a7-5aea0671c95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:02.167038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:02.218556 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 686/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Thu Jul 21 22:26:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:02.235227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10813dce-c6d4-43c6-a471-59e71a1ee368 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:02.239152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-10813dce-c6d4-43c6-a471-59e71a1ee368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:02.477026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10813dce-c6d4-43c6-a471-59e71a1ee368 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:02.535468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:02.575210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1ada1-b050-4671-aebe-8219e92043d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:02.576077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1ada1-b050-4671-aebe-8219e92043d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:02.576468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1ada1-b050-4671-aebe-8219e92043d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:02.576868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1ada1-b050-4671-aebe-8219e92043d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:02.579188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5ae1ada1-b050-4671-aebe-8219e92043d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:02.827943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1ada1-b050-4671-aebe-8219e92043d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:02.830188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1ada1-b050-4671-aebe-8219e92043d6 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:02.894421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 687/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:02.917093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a03b3-3649-49b5-be59-53ff3656a3bc None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:02.917093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a03b3-3649-49b5-be59-53ff3656a3bc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:02.917093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a03b3-3649-49b5-be59-53ff3656a3bc None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:02.928724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 687/1374] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:26:02 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:02.934993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe905a16-e8fb-4358-9d84-fad09f799004 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:02.935299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe905a16-e8fb-4358-9d84-fad09f799004 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:02.935691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe905a16-e8fb-4358-9d84-fad09f799004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:03.048246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-fe905a16-e8fb-4358-9d84-fad09f799004 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:03.049518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fe905a16-e8fb-4358-9d84-fad09f799004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:03.077740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:03.164327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 688/1375] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:26:03.210147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a03b3-3649-49b5-be59-53ff3656a3bc None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:03.212394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9f5a03b3-3649-49b5-be59-53ff3656a3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:03.366633 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a03b3-3649-49b5-be59-53ff3656a3bc None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:03.423360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:03.492183 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75f7f5-a15b-45ed-9c8c-7dce07035161 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:03.492860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75f7f5-a15b-45ed-9c8c-7dce07035161 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:03.493142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75f7f5-a15b-45ed-9c8c-7dce07035161 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:03.493444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75f7f5-a15b-45ed-9c8c-7dce07035161 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:03.495140 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cc75f7f5-a15b-45ed-9c8c-7dce07035161 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:03.588838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75f7f5-a15b-45ed-9c8c-7dce07035161 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:03.590793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75f7f5-a15b-45ed-9c8c-7dce07035161 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:03.610894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 688/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:26:05.792075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508951c-4dd5-446b-b26e-c3ae828adfd6 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:05.792992 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508951c-4dd5-446b-b26e-c3ae828adfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:05.793274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508951c-4dd5-446b-b26e-c3ae828adfd6 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:05.794651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 689/1377] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:05.800390 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfca999-a696-4029-97b3-c47969a8f3ee None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:05.800877 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfca999-a696-4029-97b3-c47969a8f3ee None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:05.801179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfca999-a696-4029-97b3-c47969a8f3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:05.879185 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7bfca999-a696-4029-97b3-c47969a8f3ee None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:05.880413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7bfca999-a696-4029-97b3-c47969a8f3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:05.889038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:06.020626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 689/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:26:06.067130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508951c-4dd5-446b-b26e-c3ae828adfd6 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:06.068696 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4508951c-4dd5-446b-b26e-c3ae828adfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:06.235014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508951c-4dd5-446b-b26e-c3ae828adfd6 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:06.238903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:06.399524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53066044-7a8a-4c91-8961-ff525ce60e85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:06.399794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53066044-7a8a-4c91-8961-ff525ce60e85 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:06.400041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53066044-7a8a-4c91-8961-ff525ce60e85 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:06.400404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53066044-7a8a-4c91-8961-ff525ce60e85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:06.402304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-53066044-7a8a-4c91-8961-ff525ce60e85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:06.557850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53066044-7a8a-4c91-8961-ff525ce60e85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:06.559664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53066044-7a8a-4c91-8961-ff525ce60e85 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:06.584708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 690/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:08.744527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae482c7-426a-42d5-be9c-2014446da081 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:08.745098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae482c7-426a-42d5-be9c-2014446da081 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:08.745098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae482c7-426a-42d5-be9c-2014446da081 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:08.746616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 690/1380] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:26:08.751921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a75d4-f781-484b-8ae9-02dcf09369b3 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:08.752358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a75d4-f781-484b-8ae9-02dcf09369b3 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:08.752666 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a75d4-f781-484b-8ae9-02dcf09369b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:08.764232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:08.825669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-3e0a75d4-f781-484b-8ae9-02dcf09369b3 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:08.826872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3e0a75d4-f781-484b-8ae9-02dcf09369b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:08.941028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 691/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:08.990557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae482c7-426a-42d5-be9c-2014446da081 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:08.992234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4ae482c7-426a-42d5-be9c-2014446da081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:09.157484 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:09.169816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae482c7-426a-42d5-be9c-2014446da081 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:09.297062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687213fa-97ca-4bbf-a60d-60fcca987653 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:09.297909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687213fa-97ca-4bbf-a60d-60fcca987653 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:09.298181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687213fa-97ca-4bbf-a60d-60fcca987653 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:09.298460 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687213fa-97ca-4bbf-a60d-60fcca987653 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:09.300152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-687213fa-97ca-4bbf-a60d-60fcca987653 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:09.400514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687213fa-97ca-4bbf-a60d-60fcca987653 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:09.402164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687213fa-97ca-4bbf-a60d-60fcca987653 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:09.423897 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:09.431478 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 691/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:11.595972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d554f55-6fa8-41f4-8814-2b265a02751f None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:11.596671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d554f55-6fa8-41f4-8814-2b265a02751f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:11.596671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d554f55-6fa8-41f4-8814-2b265a02751f None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:11.597950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 692/1383] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:11.605150 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbfaac-eadc-4cf3-ae26-06f264417216 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:11.605858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbfaac-eadc-4cf3-ae26-06f264417216 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:11.605858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cbfaac-eadc-4cf3-ae26-06f264417216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:11.689865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-04cbfaac-eadc-4cf3-ae26-06f264417216 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:11.691440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-04cbfaac-eadc-4cf3-ae26-06f264417216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:11.812807 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 692/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:11.861404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d554f55-6fa8-41f4-8814-2b265a02751f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:11.862948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9d554f55-6fa8-41f4-8814-2b265a02751f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:11.963623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:12.050385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d554f55-6fa8-41f4-8814-2b265a02751f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:12.188526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee6a16-e20d-48d1-b926-c03674d3dd58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:12.189264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee6a16-e20d-48d1-b926-c03674d3dd58 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:12.189900 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee6a16-e20d-48d1-b926-c03674d3dd58 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:12.190517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee6a16-e20d-48d1-b926-c03674d3dd58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:12.192867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-61ee6a16-e20d-48d1-b926-c03674d3dd58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:12.263747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:12.324420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee6a16-e20d-48d1-b926-c03674d3dd58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:12.326855 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee6a16-e20d-48d1-b926-c03674d3dd58 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:12.353543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 693/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:14.788914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251de2f6-60aa-4331-a0c1-ba26737a1bd7 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:14.789515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251de2f6-60aa-4331-a0c1-ba26737a1bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:14.789768 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251de2f6-60aa-4331-a0c1-ba26737a1bd7 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:14.791820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 693/1386] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:26:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:14.799000 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5209c4-edf5-4008-bf8d-c6e20da3ea9c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:14.799677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5209c4-edf5-4008-bf8d-c6e20da3ea9c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:14.799677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5209c4-edf5-4008-bf8d-c6e20da3ea9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:14.872806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-1c5209c4-edf5-4008-bf8d-c6e20da3ea9c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:14.874010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1c5209c4-edf5-4008-bf8d-c6e20da3ea9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:14.996290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 694/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:15.044423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251de2f6-60aa-4331-a0c1-ba26737a1bd7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:15.046254 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-251de2f6-60aa-4331-a0c1-ba26737a1bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:15.130930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:15.208351 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251de2f6-60aa-4331-a0c1-ba26737a1bd7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:15.348699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed749525-9e9b-43ec-92a2-4b93f69050a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:15.349329 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed749525-9e9b-43ec-92a2-4b93f69050a7 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:15.349615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed749525-9e9b-43ec-92a2-4b93f69050a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:15.349905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed749525-9e9b-43ec-92a2-4b93f69050a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:15.352223 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ed749525-9e9b-43ec-92a2-4b93f69050a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:15.393116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:15.456417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed749525-9e9b-43ec-92a2-4b93f69050a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:15.458106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed749525-9e9b-43ec-92a2-4b93f69050a7 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:15.479277 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 694/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:17.757250 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd31bb-07a0-48fe-a944-8a43c7de4629 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:17.758203 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd31bb-07a0-48fe-a944-8a43c7de4629 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:17.758579 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd31bb-07a0-48fe-a944-8a43c7de4629 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:17.760075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 695/1389] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:17.766726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf07f95-fb05-4dfc-86b5-95050d6cf8a5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:17.767236 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf07f95-fb05-4dfc-86b5-95050d6cf8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:17.767442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf07f95-fb05-4dfc-86b5-95050d6cf8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:17.844339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-eaf07f95-fb05-4dfc-86b5-95050d6cf8a5 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:17.845603 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-eaf07f95-fb05-4dfc-86b5-95050d6cf8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:17.952559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 695/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:17.997436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd31bb-07a0-48fe-a944-8a43c7de4629 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:17.998966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-79dd31bb-07a0-48fe-a944-8a43c7de4629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:18.032986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:18.169756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd31bb-07a0-48fe-a944-8a43c7de4629 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:18.290125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:18.303922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fb590-6588-490e-bd55-38270f522bcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:18.305912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fb590-6588-490e-bd55-38270f522bcb service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:18.306209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fb590-6588-490e-bd55-38270f522bcb service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:18.306504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fb590-6588-490e-bd55-38270f522bcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:18.308241 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-e84fb590-6588-490e-bd55-38270f522bcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:18.375842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0f7e8-c0bf-4562-a1af-80460e0989e7 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:18.376232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0f7e8-c0bf-4562-a1af-80460e0989e7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:18.376601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0f7e8-c0bf-4562-a1af-80460e0989e7 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:18.378446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 696/1391] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:26:18.384869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eea44e-834f-49dc-988b-35d0beccff8f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:18.385234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eea44e-834f-49dc-988b-35d0beccff8f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:18.385578 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eea44e-834f-49dc-988b-35d0beccff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:18.403066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:18.421762 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84fb590-6588-490e-bd55-38270f522bcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:18.423289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84fb590-6588-490e-bd55-38270f522bcb service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:18.447233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 696/1392] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:26:18.470058 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e3eea44e-834f-49dc-988b-35d0beccff8f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:18.471312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e3eea44e-834f-49dc-988b-35d0beccff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:18.581912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 697/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:18.642205 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03a416-2327-41f7-ad6c-a387973603ec None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:18.642524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03a416-2327-41f7-ad6c-a387973603ec None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:18.642806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03a416-2327-41f7-ad6c-a387973603ec None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:18.644239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 697/1394] 10.222.0.42 () {54 vars in 966 bytes} [Thu Jul 21 22:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:18.652192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb4844-e3fd-4f2e-92b9-bccc6de99414 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:18.652694 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb4844-e3fd-4f2e-92b9-bccc6de99414 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:18.653108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb4844-e3fd-4f2e-92b9-bccc6de99414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:18.737013 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-57eb4844-e3fd-4f2e-92b9-bccc6de99414 None None] MFA Rules not processed for user `78b67db080874f3ebb508239c7a8ccf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:18.738209 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-57eb4844-e3fd-4f2e-92b9-bccc6de99414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:18.775631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:18.820680 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 698/1395] 10.222.0.42 () {58 vars in 1063 bytes} [Thu Jul 21 22:26:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:18.828927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03a416-2327-41f7-ad6c-a387973603ec None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:18.830675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2a03a416-2327-41f7-ad6c-a387973603ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:18.950432 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03a416-2327-41f7-ad6c-a387973603ec None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:18.964825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:19.041010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc841c9-0bd0-4255-86bb-18fbcf430a82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b67db080874f3ebb508239c7a8ccf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:19.041669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc841c9-0bd0-4255-86bb-18fbcf430a82 service glance] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:19.041957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc841c9-0bd0-4255-86bb-18fbcf430a82 service glance] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:19.042238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc841c9-0bd0-4255-86bb-18fbcf430a82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:19.043975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1dc841c9-0bd0-4255-86bb-18fbcf430a82 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:19.168373 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc841c9-0bd0-4255-86bb-18fbcf430a82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:19.169871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc841c9-0bd0-4255-86bb-18fbcf430a82 service glance] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:19.189853 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 698/1396] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:19.946548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72114781-949c-4733-b2c5-c04300df4662 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:19.947060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72114781-949c-4733-b2c5-c04300df4662 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:19.947269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72114781-949c-4733-b2c5-c04300df4662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:20.018841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-72114781-949c-4733-b2c5-c04300df4662 None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:20.020151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-72114781-949c-4733-b2c5-c04300df4662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:20.114982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 699/1397] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 22:26:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:20.161931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a4779-f143-489b-a24e-29abe139953f None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:20.162424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a4779-f143-489b-a24e-29abe139953f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:20.162507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a4779-f143-489b-a24e-29abe139953f None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:20.164824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 699/1398] 10.222.0.42 () {54 vars in 943 bytes} [Thu Jul 21 22:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:20.171473 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071a8e5-d6dd-4d42-b93f-286d5a6fb2df None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:20.171775 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071a8e5-d6dd-4d42-b93f-286d5a6fb2df None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:20.172060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071a8e5-d6dd-4d42-b93f-286d5a6fb2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:20.234837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:20.255612 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7071a8e5-d6dd-4d42-b93f-286d5a6fb2df None None] MFA Rules not processed for user `8fec30ef0b8c41dca643cfefebfe904b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:20.256861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7071a8e5-d6dd-4d42-b93f-286d5a6fb2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:20.369688 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 700/1399] 10.222.0.42 () {58 vars in 1040 bytes} [Thu Jul 21 22:26:20 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:20.379976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a4779-f143-489b-a24e-29abe139953f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:20.381590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-168a4779-f143-489b-a24e-29abe139953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:20.439325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:20.533160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a4779-f143-489b-a24e-29abe139953f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:20.590302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ad6fa-f51a-49f2-a3e0-2b7025ccd509 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:20.590760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ad6fa-f51a-49f2-a3e0-2b7025ccd509 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:20.591028 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ad6fa-f51a-49f2-a3e0-2b7025ccd509 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:20.592472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 701/1400] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:26:20.598366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd95da2-ffa3-4f49-8a90-01e13efa1f94 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:20.598764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd95da2-ffa3-4f49-8a90-01e13efa1f94 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:20.599110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd95da2-ffa3-4f49-8a90-01e13efa1f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:20.643027 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56155f-49ee-4f7f-97cf-49d6dca40ce6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:20.643686 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56155f-49ee-4f7f-97cf-49d6dca40ce6 service swift] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:20.643956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56155f-49ee-4f7f-97cf-49d6dca40ce6 service swift] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:20.644225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56155f-49ee-4f7f-97cf-49d6dca40ce6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:20.645981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1b56155f-49ee-4f7f-97cf-49d6dca40ce6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:20.675798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8dd95da2-ffa3-4f49-8a90-01e13efa1f94 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:20.677039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8dd95da2-ffa3-4f49-8a90-01e13efa1f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:20.678672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:20.746312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56155f-49ee-4f7f-97cf-49d6dca40ce6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:20.747675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:20.747927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56155f-49ee-4f7f-97cf-49d6dca40ce6 service swift] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:20.757243 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 700/1401] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 22:26:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:26:20.795969 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 702/1402] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:26:20.844152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56155f-49ee-4f7f-97cf-49d6dca40ce6 service swift] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:20.846288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1b56155f-49ee-4f7f-97cf-49d6dca40ce6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:20.960613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56155f-49ee-4f7f-97cf-49d6dca40ce6 service swift] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:21.060184 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:21.137551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43074c1-c6d5-48c6-a1a9-ed7cfc4f065d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:21.138225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43074c1-c6d5-48c6-a1a9-ed7cfc4f065d service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:21.138534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43074c1-c6d5-48c6-a1a9-ed7cfc4f065d service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:21.138896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43074c1-c6d5-48c6-a1a9-ed7cfc4f065d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:21.140747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b43074c1-c6d5-48c6-a1a9-ed7cfc4f065d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:21.250552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43074c1-c6d5-48c6-a1a9-ed7cfc4f065d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:21.252171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43074c1-c6d5-48c6-a1a9-ed7cfc4f065d service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:21.274869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 701/1403] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:23.409374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833aacd4-869b-4912-9895-1f97677d7ac1 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:23.410424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833aacd4-869b-4912-9895-1f97677d7ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:23.410780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833aacd4-869b-4912-9895-1f97677d7ac1 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:23.412462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 703/1404] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:26:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:23.419211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54752889-dcf0-4797-a557-aca221cefa1b None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:23.419211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54752889-dcf0-4797-a557-aca221cefa1b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:23.419211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54752889-dcf0-4797-a557-aca221cefa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:23.489190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-54752889-dcf0-4797-a557-aca221cefa1b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:23.490410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-54752889-dcf0-4797-a557-aca221cefa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:23.532201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:23.600982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 702/1405] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:23.645378 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833aacd4-869b-4912-9895-1f97677d7ac1 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:23.650330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-833aacd4-869b-4912-9895-1f97677d7ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:23.833355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833aacd4-869b-4912-9895-1f97677d7ac1 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:23.883586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:23.996159 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0d363-8f5d-45a0-917c-b2465d861924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:23.996893 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0d363-8f5d-45a0-917c-b2465d861924 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:23.997189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0d363-8f5d-45a0-917c-b2465d861924 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:23.997466 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0d363-8f5d-45a0-917c-b2465d861924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:23.999363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4fa0d363-8f5d-45a0-917c-b2465d861924 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:24.140988 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0d363-8f5d-45a0-917c-b2465d861924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:24.142901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0d363-8f5d-45a0-917c-b2465d861924 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:24.191493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 704/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:26.349468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b330bd8-2b82-4e34-9a3a-f7ad4bf3d0eb None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:26.350051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b330bd8-2b82-4e34-9a3a-f7ad4bf3d0eb None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:26.350051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b330bd8-2b82-4e34-9a3a-f7ad4bf3d0eb None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:26.351483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 703/1407] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:26.357101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d045c-a1b7-4b53-a97e-1072dc1497d6 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:26.357593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d045c-a1b7-4b53-a97e-1072dc1497d6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:26.357694 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d045c-a1b7-4b53-a97e-1072dc1497d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:26.438589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:26.451318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-741d045c-a1b7-4b53-a97e-1072dc1497d6 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:26.452981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-741d045c-a1b7-4b53-a97e-1072dc1497d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:26.567729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 705/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:26.612900 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b330bd8-2b82-4e34-9a3a-f7ad4bf3d0eb None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:26.614583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4b330bd8-2b82-4e34-9a3a-f7ad4bf3d0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:26.787716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b330bd8-2b82-4e34-9a3a-f7ad4bf3d0eb None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:26.808929 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:26.929089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210032c-b7dc-4b9b-a4de-4f18fac26fa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:26.930074 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210032c-b7dc-4b9b-a4de-4f18fac26fa0 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:26.930361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210032c-b7dc-4b9b-a4de-4f18fac26fa0 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:26.930639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210032c-b7dc-4b9b-a4de-4f18fac26fa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:26.932392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3210032c-b7dc-4b9b-a4de-4f18fac26fa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:27.033290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210032c-b7dc-4b9b-a4de-4f18fac26fa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:27.034991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210032c-b7dc-4b9b-a4de-4f18fac26fa0 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:27.061102 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 704/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:36.565688 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=44 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:37.057731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:43.760040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2609f-8b63-46b6-88cf-4a98219db2b5 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:43.760417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2609f-8b63-46b6-88cf-4a98219db2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:43.760611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2609f-8b63-46b6-88cf-4a98219db2b5 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:43.762468 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 706/1410] 10.222.0.42 () {54 vars in 918 bytes} [Thu Jul 21 22:26:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:43.768775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3551fc-3e34-42b9-97dc-47d9e566ae38 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:43.769281 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3551fc-3e34-42b9-97dc-47d9e566ae38 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:43.769486 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3551fc-3e34-42b9-97dc-47d9e566ae38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:43.845938 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-6e3551fc-3e34-42b9-97dc-47d9e566ae38 None None] MFA Rules not processed for user `705ff0e73e95499e873b1df2a24fefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:43.847142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6e3551fc-3e34-42b9-97dc-47d9e566ae38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:43.952971 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 705/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Jul 21 22:26:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:43.967757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1169111-d458-4cc1-aa5e-d835d90e3d51 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:43.968051 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1169111-d458-4cc1-aa5e-d835d90e3d51 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:43.968335 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1169111-d458-4cc1-aa5e-d835d90e3d51 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:43.969852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 707/1412] 10.222.0.42 () {54 vars in 918 bytes} [Thu Jul 21 22:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:43.974641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaac33a-535e-4cff-a018-175bfa012946 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:43.974937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaac33a-535e-4cff-a018-175bfa012946 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:43.975811 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaac33a-535e-4cff-a018-175bfa012946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:44.046903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:44.050141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ecaac33a-535e-4cff-a018-175bfa012946 None None] MFA Rules not processed for user `c77f6ccc7cb84c04ac532007a04a0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:44.051891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ecaac33a-535e-4cff-a018-175bfa012946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:44.175634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 706/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Jul 21 22:26:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:46.720654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0e919-9ced-4172-a478-801a0312c840 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:46.721457 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0e919-9ced-4172-a478-801a0312c840 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:46.721457 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0e919-9ced-4172-a478-801a0312c840 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:46.723262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:46.729190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8fa24-118d-4cde-9fc7-eba852c2371d None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:46.729642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8fa24-118d-4cde-9fc7-eba852c2371d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:46.729755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8fa24-118d-4cde-9fc7-eba852c2371d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:46.797044 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-83d8fa24-118d-4cde-9fc7-eba852c2371d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:46.798273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-83d8fa24-118d-4cde-9fc7-eba852c2371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:46.854235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:46.897964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:46.941888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0e919-9ced-4172-a478-801a0312c840 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:46.943436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0cf0e919-9ced-4172-a478-801a0312c840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:47.100905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0e919-9ced-4172-a478-801a0312c840 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:47.175779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:47.223139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084bdf-c209-458c-b28f-b1450f80c7ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:47.223878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084bdf-c209-458c-b28f-b1450f80c7ec service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:47.224170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084bdf-c209-458c-b28f-b1450f80c7ec service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:47.224452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084bdf-c209-458c-b28f-b1450f80c7ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:47.226772 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4a084bdf-c209-458c-b28f-b1450f80c7ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:47.319595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a084bdf-c209-458c-b28f-b1450f80c7ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:47.321599 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a084bdf-c209-458c-b28f-b1450f80c7ec service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:47.340847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 709/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:48.267759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022880d2-d888-43a1-a3f1-13f0056926b0 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:48.268347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022880d2-d888-43a1-a3f1-13f0056926b0 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:48.268347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022880d2-d888-43a1-a3f1-13f0056926b0 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:48.269706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:48.275167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b99d18-be78-43ec-a30a-b06099a4a37e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:48.275662 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b99d18-be78-43ec-a30a-b06099a4a37e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:48.275818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b99d18-be78-43ec-a30a-b06099a4a37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:48.352704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-24b99d18-be78-43ec-a30a-b06099a4a37e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:48.353965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-24b99d18-be78-43ec-a30a-b06099a4a37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:48.382691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:48.459940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:26:48.616216 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022880d2-d888-43a1-a3f1-13f0056926b0 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:48.618370 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-022880d2-d888-43a1-a3f1-13f0056926b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:48.771812 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022880d2-d888-43a1-a3f1-13f0056926b0 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:48.814804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:48.894878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae317627-81e2-464f-a9cf-03f52c3c4719 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:48.895465 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae317627-81e2-464f-a9cf-03f52c3c4719 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:48.895757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae317627-81e2-464f-a9cf-03f52c3c4719 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:48.896042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae317627-81e2-464f-a9cf-03f52c3c4719 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:48.897892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ae317627-81e2-464f-a9cf-03f52c3c4719 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:48.992080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae317627-81e2-464f-a9cf-03f52c3c4719 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:48.993659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae317627-81e2-464f-a9cf-03f52c3c4719 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:49.012256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 709/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:49.916010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69abd5-2b59-49d0-957e-eca7d44e1dc0 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:49.916585 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69abd5-2b59-49d0-957e-eca7d44e1dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:49.916585 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69abd5-2b59-49d0-957e-eca7d44e1dc0 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:49.917984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:49.923612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c9d3c-eb80-4536-9a6f-e0055cb98da5 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:49.924091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c9d3c-eb80-4536-9a6f-e0055cb98da5 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:49.924189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c9d3c-eb80-4536-9a6f-e0055cb98da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:49.978299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:50.000617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-755c9d3c-eb80-4536-9a6f-e0055cb98da5 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:50.002024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-755c9d3c-eb80-4536-9a6f-e0055cb98da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:50.104321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:50.258438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69abd5-2b59-49d0-957e-eca7d44e1dc0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:50.259978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0e69abd5-2b59-49d0-957e-eca7d44e1dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:50.409253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69abd5-2b59-49d0-957e-eca7d44e1dc0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:50.423717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:50.531255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf25e8a-3741-416f-a8cb-089db5c29abc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:50.531829 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf25e8a-3741-416f-a8cb-089db5c29abc service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:50.532112 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf25e8a-3741-416f-a8cb-089db5c29abc service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:50.532388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf25e8a-3741-416f-a8cb-089db5c29abc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:50.534143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9bf25e8a-3741-416f-a8cb-089db5c29abc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:50.628194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf25e8a-3741-416f-a8cb-089db5c29abc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:50.629876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf25e8a-3741-416f-a8cb-089db5c29abc service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:50.649927 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 712/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:51.537112 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d3a4d-e18d-4f6c-8cca-5ee9d42f971a None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:51.537645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d3a4d-e18d-4f6c-8cca-5ee9d42f971a None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:51.537771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d3a4d-e18d-4f6c-8cca-5ee9d42f971a None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:51.539188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:51.544185 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e5854-d962-496e-8695-166be82a0bb1 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:51.544631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e5854-d962-496e-8695-166be82a0bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:51.544782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e5854-d962-496e-8695-166be82a0bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:51.564246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:51.621314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e72e5854-d962-496e-8695-166be82a0bb1 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:51.622550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e72e5854-d962-496e-8695-166be82a0bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:51.726343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:51.878739 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d3a4d-e18d-4f6c-8cca-5ee9d42f971a None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:51.880275 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-536d3a4d-e18d-4f6c-8cca-5ee9d42f971a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:51.999947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:52.043247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d3a4d-e18d-4f6c-8cca-5ee9d42f971a None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:52.160062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e399600-240b-48ce-abe8-7bcad640a778 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:52.160672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e399600-240b-48ce-abe8-7bcad640a778 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:52.160958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e399600-240b-48ce-abe8-7bcad640a778 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:52.161237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e399600-240b-48ce-abe8-7bcad640a778 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:52.162917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9e399600-240b-48ce-abe8-7bcad640a778 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:52.242588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:52.256780 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e399600-240b-48ce-abe8-7bcad640a778 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:52.258843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e399600-240b-48ce-abe8-7bcad640a778 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:52.278583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 712/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:53.191847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb50e8d-c1f2-4884-a184-07faa0218c31 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:53.192437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb50e8d-c1f2-4884-a184-07faa0218c31 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:53.192437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb50e8d-c1f2-4884-a184-07faa0218c31 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:53.193802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:53.200236 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b1ec3-de2f-4301-bdd7-2e7b378fcec1 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:53.200721 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b1ec3-de2f-4301-bdd7-2e7b378fcec1 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:53.200866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b1ec3-de2f-4301-bdd7-2e7b378fcec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:53.287230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-c97b1ec3-de2f-4301-bdd7-2e7b378fcec1 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:53.288527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c97b1ec3-de2f-4301-bdd7-2e7b378fcec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:53.395176 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:53.550169 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb50e8d-c1f2-4884-a184-07faa0218c31 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:53.551741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cdb50e8d-c1f2-4884-a184-07faa0218c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:53.637684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:53.705448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb50e8d-c1f2-4884-a184-07faa0218c31 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:53.830330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f46127-460f-4365-aad0-8de32349e681 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:53.830898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f46127-460f-4365-aad0-8de32349e681 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:53.831186 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f46127-460f-4365-aad0-8de32349e681 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:53.831464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f46127-460f-4365-aad0-8de32349e681 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:53.833180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c3f46127-460f-4365-aad0-8de32349e681 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:53.881536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:53.924201 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f46127-460f-4365-aad0-8de32349e681 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:53.925874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f46127-460f-4365-aad0-8de32349e681 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:53.944863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 715/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:54.784306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36059d-6627-4f6a-ba10-a0c7cd8865f5 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:54.784895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36059d-6627-4f6a-ba10-a0c7cd8865f5 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:54.784895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36059d-6627-4f6a-ba10-a0c7cd8865f5 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:54.786300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:54.791495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99fa09-127a-44c2-9ed2-531a5f9cc0b8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:54.791964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99fa09-127a-44c2-9ed2-531a5f9cc0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:54.792066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99fa09-127a-44c2-9ed2-531a5f9cc0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:54.867503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0d99fa09-127a-44c2-9ed2-531a5f9cc0b8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:54.868786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0d99fa09-127a-44c2-9ed2-531a5f9cc0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:54.966446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:55.117794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36059d-6627-4f6a-ba10-a0c7cd8865f5 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:55.119260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5f36059d-6627-4f6a-ba10-a0c7cd8865f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:55.169482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:55.273107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36059d-6627-4f6a-ba10-a0c7cd8865f5 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:55.388700 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba28dd-be36-4c4b-baf5-2c5a2902fbf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:55.389264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba28dd-be36-4c4b-baf5-2c5a2902fbf9 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:55.389542 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba28dd-be36-4c4b-baf5-2c5a2902fbf9 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:55.389817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba28dd-be36-4c4b-baf5-2c5a2902fbf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:55.391477 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-80ba28dd-be36-4c4b-baf5-2c5a2902fbf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:55.398765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:55.488263 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba28dd-be36-4c4b-baf5-2c5a2902fbf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:55.489920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba28dd-be36-4c4b-baf5-2c5a2902fbf9 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:55.509835 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 715/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:56.327723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf59807-2345-4134-84a0-a05dbbec02d8 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:56.328316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf59807-2345-4134-84a0-a05dbbec02d8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:56.328316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf59807-2345-4134-84a0-a05dbbec02d8 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:56.329693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:56.334702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3db110-5733-4aae-b905-0f10e07b8a97 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:56.335130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3db110-5733-4aae-b905-0f10e07b8a97 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:56.335317 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3db110-5733-4aae-b905-0f10e07b8a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:56.412645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7e3db110-5733-4aae-b905-0f10e07b8a97 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:56.414900 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7e3db110-5733-4aae-b905-0f10e07b8a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:56.485441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:56.516557 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:26:56.669259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf59807-2345-4134-84a0-a05dbbec02d8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:56.670806 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8cf59807-2345-4134-84a0-a05dbbec02d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:56.688808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:56.824876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf59807-2345-4134-84a0-a05dbbec02d8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:56.914546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:56.941198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1235de4-142d-486b-bd9d-7b2694d22292 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:56.941765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1235de4-142d-486b-bd9d-7b2694d22292 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:56.942044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1235de4-142d-486b-bd9d-7b2694d22292 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:56.942321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1235de4-142d-486b-bd9d-7b2694d22292 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:56.943967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d1235de4-142d-486b-bd9d-7b2694d22292 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:57.040028 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1235de4-142d-486b-bd9d-7b2694d22292 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:57.041716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1235de4-142d-486b-bd9d-7b2694d22292 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:57.059975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 718/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:57.878066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07592ec3-e84f-44e9-a80b-86c4fbefe5ef None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:57.878612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07592ec3-e84f-44e9-a80b-86c4fbefe5ef None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:57.878612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07592ec3-e84f-44e9-a80b-86c4fbefe5ef None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:57.879982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:57.885813 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9772da-e855-4a23-ac63-347994e7a25e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:57.886241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9772da-e855-4a23-ac63-347994e7a25e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:57.886409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9772da-e855-4a23-ac63-347994e7a25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:57.962827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8c9772da-e855-4a23-ac63-347994e7a25e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:57.964064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8c9772da-e855-4a23-ac63-347994e7a25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:58.005697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:58.066115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:58.221494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07592ec3-e84f-44e9-a80b-86c4fbefe5ef None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:58.222960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-07592ec3-e84f-44e9-a80b-86c4fbefe5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:58.377932 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07592ec3-e84f-44e9-a80b-86c4fbefe5ef None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:58.434379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:58.493165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ddc17-d172-4681-9c98-42b218b685f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:26:58.493737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ddc17-d172-4681-9c98-42b218b685f8 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:58.494026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ddc17-d172-4681-9c98-42b218b685f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:58.494301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ddc17-d172-4681-9c98-42b218b685f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:58.495947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a65ddc17-d172-4681-9c98-42b218b685f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:58.584054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ddc17-d172-4681-9c98-42b218b685f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:26:58.585969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ddc17-d172-4681-9c98-42b218b685f8 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:26:58.605022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 718/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:26:59.426008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf98783-1421-4d49-b0fb-39a612ae0e72 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:59.426607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf98783-1421-4d49-b0fb-39a612ae0e72 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:59.426607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf98783-1421-4d49-b0fb-39a612ae0e72 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:59.428020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:26:59.435256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba452934-9003-4d8c-9bb2-bc2439bcb949 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:26:59.435666 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba452934-9003-4d8c-9bb2-bc2439bcb949 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:26:59.435955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba452934-9003-4d8c-9bb2-bc2439bcb949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:26:59.514186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ba452934-9003-4d8c-9bb2-bc2439bcb949 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:26:59.515414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ba452934-9003-4d8c-9bb2-bc2439bcb949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:59.517674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:26:59.620029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:26:59.769338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf98783-1421-4d49-b0fb-39a612ae0e72 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:26:59.771071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4cf98783-1421-4d49-b0fb-39a612ae0e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:26:59.919640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf98783-1421-4d49-b0fb-39a612ae0e72 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:26:59.947093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:00.034427 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a5c0a-8372-4bd3-ba6d-aec305d0fe84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:00.035089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a5c0a-8372-4bd3-ba6d-aec305d0fe84 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:00.035375 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a5c0a-8372-4bd3-ba6d-aec305d0fe84 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:00.035658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a5c0a-8372-4bd3-ba6d-aec305d0fe84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:00.037436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bc3a5c0a-8372-4bd3-ba6d-aec305d0fe84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:00.129646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a5c0a-8372-4bd3-ba6d-aec305d0fe84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:00.131268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a5c0a-8372-4bd3-ba6d-aec305d0fe84 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:00.151679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 721/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:02.181691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3d5ce-f5d7-415d-adea-ea2c2d7ef812 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:02.182254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3d5ce-f5d7-415d-adea-ea2c2d7ef812 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:02.182254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3d5ce-f5d7-415d-adea-ea2c2d7ef812 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:02.183613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:02.188967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8d3d8-8d2f-4c78-9a4c-11000f4389cb None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:02.189394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8d3d8-8d2f-4c78-9a4c-11000f4389cb None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:02.189532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8d3d8-8d2f-4c78-9a4c-11000f4389cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:02.233960 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:02.260140 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a3d8d3d8-8d2f-4c78-9a4c-11000f4389cb None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:02.261418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a3d8d3d8-8d2f-4c78-9a4c-11000f4389cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:02.367996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:02.407453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3d5ce-f5d7-415d-adea-ea2c2d7ef812 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:02.409755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c9e3d5ce-f5d7-415d-adea-ea2c2d7ef812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:02.521455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3d5ce-f5d7-415d-adea-ea2c2d7ef812 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:02.555941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:02.597925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33840ec6-b783-4673-bde8-8a22f27de023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:02.598588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33840ec6-b783-4673-bde8-8a22f27de023 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:02.598880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33840ec6-b783-4673-bde8-8a22f27de023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:02.599163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33840ec6-b783-4673-bde8-8a22f27de023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:02.601749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-33840ec6-b783-4673-bde8-8a22f27de023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:02.714918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33840ec6-b783-4673-bde8-8a22f27de023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:02.716302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33840ec6-b783-4673-bde8-8a22f27de023 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:02.734286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 721/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:05.033148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ccb81-2990-42d2-b678-83b2d63e2c88 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:05.033741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ccb81-2990-42d2-b678-83b2d63e2c88 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:05.033741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ccb81-2990-42d2-b678-83b2d63e2c88 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:05.035080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:05.040787 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b85f63-af5a-43cc-ad95-0ea904ca4045 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:05.041286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b85f63-af5a-43cc-ad95-0ea904ca4045 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:05.041385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b85f63-af5a-43cc-ad95-0ea904ca4045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:05.108610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-63b85f63-af5a-43cc-ad95-0ea904ca4045 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:05.109810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-63b85f63-af5a-43cc-ad95-0ea904ca4045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:05.123212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:05.211289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:05.251056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ccb81-2990-42d2-b678-83b2d63e2c88 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:05.252682 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e46ccb81-2990-42d2-b678-83b2d63e2c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:05.365194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ccb81-2990-42d2-b678-83b2d63e2c88 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:05.378021 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:05.443897 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff6e4a-c166-45af-8e42-b5284bd4227d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:05.444655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff6e4a-c166-45af-8e42-b5284bd4227d service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:05.444947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff6e4a-c166-45af-8e42-b5284bd4227d service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:05.445255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff6e4a-c166-45af-8e42-b5284bd4227d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:05.446960 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-39ff6e4a-c166-45af-8e42-b5284bd4227d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:05.557464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff6e4a-c166-45af-8e42-b5284bd4227d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:05.558916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff6e4a-c166-45af-8e42-b5284bd4227d service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:05.577504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 724/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:07.996353 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840ec5f-b1c0-439d-8ffe-ab2663a1df7c None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:07.996930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840ec5f-b1c0-439d-8ffe-ab2663a1df7c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:07.996930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840ec5f-b1c0-439d-8ffe-ab2663a1df7c None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:07.998318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:08.004093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaea2e2-6bfd-4073-92e8-93c034f55782 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:08.004544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaea2e2-6bfd-4073-92e8-93c034f55782 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:08.004660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaea2e2-6bfd-4073-92e8-93c034f55782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:08.051525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:08.070526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-efaea2e2-6bfd-4073-92e8-93c034f55782 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:08.071774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-efaea2e2-6bfd-4073-92e8-93c034f55782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:08.168742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:08.207272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840ec5f-b1c0-439d-8ffe-ab2663a1df7c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:08.208926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d840ec5f-b1c0-439d-8ffe-ab2663a1df7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:08.314947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840ec5f-b1c0-439d-8ffe-ab2663a1df7c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:08.356287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:08.391021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c516cd-53c5-4560-8e88-0cafde02af24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:08.391674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c516cd-53c5-4560-8e88-0cafde02af24 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:08.391963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c516cd-53c5-4560-8e88-0cafde02af24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:08.392240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c516cd-53c5-4560-8e88-0cafde02af24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:08.393951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-44c516cd-53c5-4560-8e88-0cafde02af24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:08.505477 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c516cd-53c5-4560-8e88-0cafde02af24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:08.507469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c516cd-53c5-4560-8e88-0cafde02af24 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:08.525714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 724/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:10.744834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700f88f-0083-4604-b054-7bcf552d97d7 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:10.745374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700f88f-0083-4604-b054-7bcf552d97d7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:10.745374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700f88f-0083-4604-b054-7bcf552d97d7 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:10.746701 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:10.752072 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca497a6-c5ad-4c10-b1c6-e441615db813 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:10.752545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca497a6-c5ad-4c10-b1c6-e441615db813 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:10.752673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca497a6-c5ad-4c10-b1c6-e441615db813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:10.821133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-eca497a6-c5ad-4c10-b1c6-e441615db813 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:10.822367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-eca497a6-c5ad-4c10-b1c6-e441615db813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:10.843703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:10.922867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:10.962698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700f88f-0083-4604-b054-7bcf552d97d7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:10.964308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0700f88f-0083-4604-b054-7bcf552d97d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:11.070321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700f88f-0083-4604-b054-7bcf552d97d7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:11.092236 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:11.149304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c212c08-e55a-4dc7-ac4b-2bb44ab12158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:11.149964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c212c08-e55a-4dc7-ac4b-2bb44ab12158 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:11.150250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c212c08-e55a-4dc7-ac4b-2bb44ab12158 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:11.150527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c212c08-e55a-4dc7-ac4b-2bb44ab12158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:11.152284 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4c212c08-e55a-4dc7-ac4b-2bb44ab12158 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:11.263265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c212c08-e55a-4dc7-ac4b-2bb44ab12158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:11.264756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c212c08-e55a-4dc7-ac4b-2bb44ab12158 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:11.283469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 727/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:13.410775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52940af0-f7e9-4058-928b-3266cf562b39 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:13.411348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52940af0-f7e9-4058-928b-3266cf562b39 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:13.411348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52940af0-f7e9-4058-928b-3266cf562b39 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:13.412777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:13.419808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b16f4d-49e3-4c84-a044-e5b293f3336f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:13.420312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b16f4d-49e3-4c84-a044-e5b293f3336f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:13.420411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b16f4d-49e3-4c84-a044-e5b293f3336f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:13.484084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:13.487086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-25b16f4d-49e3-4c84-a044-e5b293f3336f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:13.489012 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-25b16f4d-49e3-4c84-a044-e5b293f3336f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:13.584598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:13.624710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52940af0-f7e9-4058-928b-3266cf562b39 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:13.626262 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-52940af0-f7e9-4058-928b-3266cf562b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:13.737443 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52940af0-f7e9-4058-928b-3266cf562b39 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:13.790064 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:13.814879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfa58e-40de-4214-b219-921b17b05b5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:13.815536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfa58e-40de-4214-b219-921b17b05b5d service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:13.815827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfa58e-40de-4214-b219-921b17b05b5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:13.816108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfa58e-40de-4214-b219-921b17b05b5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:13.817970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-94bfa58e-40de-4214-b219-921b17b05b5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:13.940790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfa58e-40de-4214-b219-921b17b05b5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:13.942250 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bfa58e-40de-4214-b219-921b17b05b5d service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:13.967810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 727/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:16.095667 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4044c-ba6c-4cf7-8397-a457b8ae99b4 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:16.096211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4044c-ba6c-4cf7-8397-a457b8ae99b4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:16.096211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4044c-ba6c-4cf7-8397-a457b8ae99b4 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:16.097636 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:16.103153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb4194-f51a-438a-9e44-34ced2359d21 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:16.103704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb4194-f51a-438a-9e44-34ced2359d21 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:16.103704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb4194-f51a-438a-9e44-34ced2359d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:16.177614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-2feb4194-f51a-438a-9e44-34ced2359d21 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:16.179603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2feb4194-f51a-438a-9e44-34ced2359d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:16.209251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:16.287479 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:16.326280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4044c-ba6c-4cf7-8397-a457b8ae99b4 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:16.327851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c4e4044c-ba6c-4cf7-8397-a457b8ae99b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:16.445570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4044c-ba6c-4cf7-8397-a457b8ae99b4 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:16.475171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:16.524862 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bb555-f534-4794-8f2f-43ce9b6570f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:16.525520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bb555-f534-4794-8f2f-43ce9b6570f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:16.525802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bb555-f534-4794-8f2f-43ce9b6570f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:16.526078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bb555-f534-4794-8f2f-43ce9b6570f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:16.527794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-691bb555-f534-4794-8f2f-43ce9b6570f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:16.641923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bb555-f534-4794-8f2f-43ce9b6570f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:16.643384 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bb555-f534-4794-8f2f-43ce9b6570f8 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:16.666800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 730/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:18.791284 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc677c3-a3cb-4a12-88bb-14891f2b2768 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:18.791849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc677c3-a3cb-4a12-88bb-14891f2b2768 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:18.791849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc677c3-a3cb-4a12-88bb-14891f2b2768 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:18.793275 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:18.798704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d541227-23d8-4ebd-a67f-8e04add1460c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:18.799103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d541227-23d8-4ebd-a67f-8e04add1460c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:18.799269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d541227-23d8-4ebd-a67f-8e04add1460c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:18.867969 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0d541227-23d8-4ebd-a67f-8e04add1460c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:18.869231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0d541227-23d8-4ebd-a67f-8e04add1460c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:18.877640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:18.968704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:19.006920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc677c3-a3cb-4a12-88bb-14891f2b2768 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:19.008467 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bfc677c3-a3cb-4a12-88bb-14891f2b2768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:19.114237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc677c3-a3cb-4a12-88bb-14891f2b2768 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:19.174651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:19.194407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03c016-3c09-4642-b7fc-ad544c45c42a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:19.195216 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03c016-3c09-4642-b7fc-ad544c45c42a service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:19.195502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03c016-3c09-4642-b7fc-ad544c45c42a service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:19.195779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03c016-3c09-4642-b7fc-ad544c45c42a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:19.197507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-be03c016-3c09-4642-b7fc-ad544c45c42a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:19.315959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03c016-3c09-4642-b7fc-ad544c45c42a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:19.317434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03c016-3c09-4642-b7fc-ad544c45c42a service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:19.336976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 730/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:21.454300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283886f-417c-4d5e-aab5-1f2761e60568 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:21.454857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283886f-417c-4d5e-aab5-1f2761e60568 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:21.454857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283886f-417c-4d5e-aab5-1f2761e60568 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:21.456235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:21.461854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea22b7-9b3f-446a-b7d1-a74868fe0241 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:21.462303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea22b7-9b3f-446a-b7d1-a74868fe0241 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:21.462430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea22b7-9b3f-446a-b7d1-a74868fe0241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:21.528028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-daea22b7-9b3f-446a-b7d1-a74868fe0241 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:21.529290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-daea22b7-9b3f-446a-b7d1-a74868fe0241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:21.568856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:21.629402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:21.668500 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283886f-417c-4d5e-aab5-1f2761e60568 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:21.670078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8283886f-417c-4d5e-aab5-1f2761e60568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:21.777722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8283886f-417c-4d5e-aab5-1f2761e60568 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:21.817161 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:21.855660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0766151-d4e0-4e18-b3d1-e8f0a1b691df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:21.856312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0766151-d4e0-4e18-b3d1-e8f0a1b691df service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:21.856625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0766151-d4e0-4e18-b3d1-e8f0a1b691df service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:21.856902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0766151-d4e0-4e18-b3d1-e8f0a1b691df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:21.858583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a0766151-d4e0-4e18-b3d1-e8f0a1b691df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:21.988033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0766151-d4e0-4e18-b3d1-e8f0a1b691df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:21.989567 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0766151-d4e0-4e18-b3d1-e8f0a1b691df service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:22.009796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 733/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:24.126535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3405baa-5d98-4354-9158-e9f5ed098b83 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:24.127183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3405baa-5d98-4354-9158-e9f5ed098b83 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:24.127183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3405baa-5d98-4354-9158-e9f5ed098b83 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:24.128735 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:24.134791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7733d7-1e04-46b6-b056-316fa41ad169 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:24.135265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7733d7-1e04-46b6-b056-316fa41ad169 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:24.135370 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7733d7-1e04-46b6-b056-316fa41ad169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:24.207162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3e7733d7-1e04-46b6-b056-316fa41ad169 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:24.208420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3e7733d7-1e04-46b6-b056-316fa41ad169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:24.227703 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:24.308123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:24.345503 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3405baa-5d98-4354-9158-e9f5ed098b83 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:24.347257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a3405baa-5d98-4354-9158-e9f5ed098b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:24.455137 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3405baa-5d98-4354-9158-e9f5ed098b83 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:24.526604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:24.534181 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5012335-2bf8-40dc-a51d-2d090b003501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:24.534851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5012335-2bf8-40dc-a51d-2d090b003501 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:24.535143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5012335-2bf8-40dc-a51d-2d090b003501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:24.535428 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5012335-2bf8-40dc-a51d-2d090b003501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:24.537218 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d5012335-2bf8-40dc-a51d-2d090b003501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:24.650456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5012335-2bf8-40dc-a51d-2d090b003501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:24.651905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5012335-2bf8-40dc-a51d-2d090b003501 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:24.670860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 733/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:26.795583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbd3ec-b767-4d5d-a90d-aa7bb87b2821 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:26.796138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbd3ec-b767-4d5d-a90d-aa7bb87b2821 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:26.796138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbd3ec-b767-4d5d-a90d-aa7bb87b2821 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:26.797557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:26.802961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b58f7b-88c1-457a-a658-cefa57d0b79f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:26.803527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b58f7b-88c1-457a-a658-cefa57d0b79f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:26.803527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b58f7b-88c1-457a-a658-cefa57d0b79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:26.870312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f8b58f7b-88c1-457a-a658-cefa57d0b79f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:26.871537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f8b58f7b-88c1-457a-a658-cefa57d0b79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:26.921907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:26.974656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:27.014105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbd3ec-b767-4d5d-a90d-aa7bb87b2821 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:27.015653 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c7dbd3ec-b767-4d5d-a90d-aa7bb87b2821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:27.121549 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbd3ec-b767-4d5d-a90d-aa7bb87b2821 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:27.170283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:27.199545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd789a-ef21-4697-bba8-f9c8dee40bd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:27.200197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd789a-ef21-4697-bba8-f9c8dee40bd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:27.200520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd789a-ef21-4697-bba8-f9c8dee40bd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:27.200802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd789a-ef21-4697-bba8-f9c8dee40bd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:27.202464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7ebd789a-ef21-4697-bba8-f9c8dee40bd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:27.313976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd789a-ef21-4697-bba8-f9c8dee40bd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:27.315411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd789a-ef21-4697-bba8-f9c8dee40bd1 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:27.335155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 736/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:36.972715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=22 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:37.332946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=66 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:47.539580 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c6685-463b-4f6e-b374-357947e934a3 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:47.540146 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c6685-463b-4f6e-b374-357947e934a3 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:47.540146 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c6685-463b-4f6e-b374-357947e934a3 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:47.541548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 735/1471] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:47.546908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a7ce7-9df8-4011-903b-df2508104ff1 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:47.547330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a7ce7-9df8-4011-903b-df2508104ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:47.547445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a7ce7-9df8-4011-903b-df2508104ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:47.616072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8a0a7ce7-9df8-4011-903b-df2508104ff1 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:47.617420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8a0a7ce7-9df8-4011-903b-df2508104ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:47.719512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 737/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:27:47.807439 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39069b-a1e1-4b7d-899d-8447d1d629b9 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:47.807725 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39069b-a1e1-4b7d-899d-8447d1d629b9 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:47.808003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39069b-a1e1-4b7d-899d-8447d1d629b9 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:47.809417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 736/1473] 10.222.0.42 () {54 vars in 966 bytes} [Thu Jul 21 22:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:47.814100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c8120-aa2c-429f-b64d-ba36ee66b40b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:47.814383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c8120-aa2c-429f-b64d-ba36ee66b40b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:47.814659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c8120-aa2c-429f-b64d-ba36ee66b40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:47.890826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e90c8120-aa2c-429f-b64d-ba36ee66b40b None None] MFA Rules not processed for user `78b67db080874f3ebb508239c7a8ccf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:47.892056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e90c8120-aa2c-429f-b64d-ba36ee66b40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:47.913671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:47.971431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 738/1474] 10.222.0.42 () {58 vars in 1063 bytes} [Thu Jul 21 22:27:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:47.979999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39069b-a1e1-4b7d-899d-8447d1d629b9 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:47.981567 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5c39069b-a1e1-4b7d-899d-8447d1d629b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:48.085798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39069b-a1e1-4b7d-899d-8447d1d629b9 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:48.159911 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff237b-326c-4cec-a9ed-5fbff2411438 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b67db080874f3ebb508239c7a8ccf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:48.160617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff237b-326c-4cec-a9ed-5fbff2411438 service glance] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:48.160907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff237b-326c-4cec-a9ed-5fbff2411438 service glance] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:48.161266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff237b-326c-4cec-a9ed-5fbff2411438 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:48.163664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-beff237b-326c-4cec-a9ed-5fbff2411438 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:48.218606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:48.282104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff237b-326c-4cec-a9ed-5fbff2411438 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:48.283464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff237b-326c-4cec-a9ed-5fbff2411438 service glance] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:48.303532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 737/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:50.463842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d028b8a-e3a2-44ad-b981-b79c3d54dc31 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:50.464395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d028b8a-e3a2-44ad-b981-b79c3d54dc31 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:50.464395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d028b8a-e3a2-44ad-b981-b79c3d54dc31 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:50.465834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 739/1476] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:50.471142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea599557-a9c6-4bc2-9a9f-6c7a55e3491f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:50.471603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea599557-a9c6-4bc2-9a9f-6c7a55e3491f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:50.471857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea599557-a9c6-4bc2-9a9f-6c7a55e3491f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:50.541989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ea599557-a9c6-4bc2-9a9f-6c7a55e3491f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:50.543239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ea599557-a9c6-4bc2-9a9f-6c7a55e3491f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:50.645287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 738/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:50.669595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d028b8a-e3a2-44ad-b981-b79c3d54dc31 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:50.671218 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9d028b8a-e3a2-44ad-b981-b79c3d54dc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:50.805655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d028b8a-e3a2-44ad-b981-b79c3d54dc31 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:50.857068 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:50.903865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06e938-4ee5-4c8e-9494-810000dbbc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:50.904555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06e938-4ee5-4c8e-9494-810000dbbc99 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:50.904840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06e938-4ee5-4c8e-9494-810000dbbc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:50.905113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06e938-4ee5-4c8e-9494-810000dbbc99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:50.905846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06e938-4ee5-4c8e-9494-810000dbbc99 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:50.908066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-0d06e938-4ee5-4c8e-9494-810000dbbc99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:27:50.908066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: warnings.warn(msg) Jul 21 22:27:50.908066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: Jul 21 22:27:50.909560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06e938-4ee5-4c8e-9494-810000dbbc99 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:50.929968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 740/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Thu Jul 21 22:27:50 2022] GET /identity/v3/projects => generated 2553 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:27:53.102397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1c281-2222-4b5a-88f8-4533fdbb5625 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:53.102972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1c281-2222-4b5a-88f8-4533fdbb5625 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:53.102972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1c281-2222-4b5a-88f8-4533fdbb5625 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:53.104325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 739/1479] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:53.109794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99748213-c3a8-4830-b630-933e574ba557 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:53.110207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99748213-c3a8-4830-b630-933e574ba557 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:53.110363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99748213-c3a8-4830-b630-933e574ba557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:53.183574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-99748213-c3a8-4830-b630-933e574ba557 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:53.184855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-99748213-c3a8-4830-b630-933e574ba557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:53.284104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 741/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:53.329040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1c281-2222-4b5a-88f8-4533fdbb5625 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:53.330598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-53d1c281-2222-4b5a-88f8-4533fdbb5625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:53.341307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:53.485085 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1c281-2222-4b5a-88f8-4533fdbb5625 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:53.569682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:53.604579 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb9f8d-f539-4206-a790-21385fe9f214 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:53.605249 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb9f8d-f539-4206-a790-21385fe9f214 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:53.605537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb9f8d-f539-4206-a790-21385fe9f214 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:53.605818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb9f8d-f539-4206-a790-21385fe9f214 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:53.607561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2ebb9f8d-f539-4206-a790-21385fe9f214 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:53.704924 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb9f8d-f539-4206-a790-21385fe9f214 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:53.706471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb9f8d-f539-4206-a790-21385fe9f214 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:53.729097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 740/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:56.011675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a14af-b5de-46df-97c0-59488bc54fab None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:56.012260 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a14af-b5de-46df-97c0-59488bc54fab None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:56.012260 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a14af-b5de-46df-97c0-59488bc54fab None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:56.013919 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 742/1482] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:27:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:56.019593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018f1c5-61b0-459c-a0b9-65b0060e86c9 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:56.020093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018f1c5-61b0-459c-a0b9-65b0060e86c9 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:56.020204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018f1c5-61b0-459c-a0b9-65b0060e86c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:56.089379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-c018f1c5-61b0-459c-a0b9-65b0060e86c9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:56.090609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c018f1c5-61b0-459c-a0b9-65b0060e86c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:56.139097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:56.191696 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 741/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:56.228704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a14af-b5de-46df-97c0-59488bc54fab None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:56.230279 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b81a14af-b5de-46df-97c0-59488bc54fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:56.254716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:56.342912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a14af-b5de-46df-97c0-59488bc54fab None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:56.418705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc00dab-6dfd-43a1-ab3a-613a91974ff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:56.419350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc00dab-6dfd-43a1-ab3a-613a91974ff9 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:56.419638 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc00dab-6dfd-43a1-ab3a-613a91974ff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:56.419933 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc00dab-6dfd-43a1-ab3a-613a91974ff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:56.421632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1bc00dab-6dfd-43a1-ab3a-613a91974ff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:56.496806 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:56.534594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc00dab-6dfd-43a1-ab3a-613a91974ff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:56.536051 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc00dab-6dfd-43a1-ab3a-613a91974ff9 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:56.557456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 743/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:27:58.657603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063a805-5eca-41d2-83ba-1f196e78ee3a None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:58.658435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063a805-5eca-41d2-83ba-1f196e78ee3a None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:58.658627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063a805-5eca-41d2-83ba-1f196e78ee3a None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:58.660030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 742/1485] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:27:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:27:58.666024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce14cf7-6d44-4600-bb95-ee033bbf289e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:58.666493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce14cf7-6d44-4600-bb95-ee033bbf289e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:58.666601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce14cf7-6d44-4600-bb95-ee033bbf289e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:58.735614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1ce14cf7-6d44-4600-bb95-ee033bbf289e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:27:58.736786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1ce14cf7-6d44-4600-bb95-ee033bbf289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:58.835174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 744/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:27:58.875200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063a805-5eca-41d2-83ba-1f196e78ee3a None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:27:58.876783 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3063a805-5eca-41d2-83ba-1f196e78ee3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:58.982580 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063a805-5eca-41d2-83ba-1f196e78ee3a None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:27:59.057993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:27:59.063340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03365c-3442-4e61-bee7-70eaf5cf675e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:27:59.063340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03365c-3442-4e61-bee7-70eaf5cf675e service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:27:59.063340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03365c-3442-4e61-bee7-70eaf5cf675e service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:27:59.063340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03365c-3442-4e61-bee7-70eaf5cf675e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:27:59.065907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3f03365c-3442-4e61-bee7-70eaf5cf675e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:27:59.177561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03365c-3442-4e61-bee7-70eaf5cf675e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:27:59.179087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03365c-3442-4e61-bee7-70eaf5cf675e service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:27:59.197658 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 743/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:28:01.514486 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a2f9a-f24c-47fa-936c-e52094f38f1b None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:28:01.515043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a2f9a-f24c-47fa-936c-e52094f38f1b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:28:01.515043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a2f9a-f24c-47fa-936c-e52094f38f1b None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:28:01.516421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 745/1488] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:28:01.522364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2e2f1-bdd1-4d51-b0be-441f127cbacf None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:28:01.522851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2e2f1-bdd1-4d51-b0be-441f127cbacf None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:28:01.522959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2e2f1-bdd1-4d51-b0be-441f127cbacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:28:01.587939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-9ed2e2f1-bdd1-4d51-b0be-441f127cbacf None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:28:01.589220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9ed2e2f1-bdd1-4d51-b0be-441f127cbacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:28:01.644715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:28:01.690150 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 744/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:28:01.740501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a2f9a-f24c-47fa-936c-e52094f38f1b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:28:01.742224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-314a2f9a-f24c-47fa-936c-e52094f38f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:28:01.779112 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:28:01.847868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a2f9a-f24c-47fa-936c-e52094f38f1b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:28:01.921106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f90b9c-b28b-4fba-b662-151d1b87c31b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:28:01.921766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f90b9c-b28b-4fba-b662-151d1b87c31b service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:28:01.922045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f90b9c-b28b-4fba-b662-151d1b87c31b service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:28:01.922321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f90b9c-b28b-4fba-b662-151d1b87c31b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:28:01.924016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-62f90b9c-b28b-4fba-b662-151d1b87c31b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:28:02.006691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:28:02.035987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f90b9c-b28b-4fba-b662-151d1b87c31b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:28:02.037999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f90b9c-b28b-4fba-b662-151d1b87c31b service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:28:02.056843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 746/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:28:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:28:04.356086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde2f94-08af-4d16-871b-17b955d7f674 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:28:04.356706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde2f94-08af-4d16-871b-17b955d7f674 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:28:04.356706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde2f94-08af-4d16-871b-17b955d7f674 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:28:04.358071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 745/1491] 10.222.0.42 () {54 vars in 926 bytes} [Thu Jul 21 22:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:28:04.363629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29c51-45d2-40e3-a46a-dbcfe7440982 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:28:04.364044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29c51-45d2-40e3-a46a-dbcfe7440982 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:28:04.364161 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29c51-45d2-40e3-a46a-dbcfe7440982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:28:04.432383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5db29c51-45d2-40e3-a46a-dbcfe7440982 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:28:04.433641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5db29c51-45d2-40e3-a46a-dbcfe7440982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:28:04.532876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 747/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Thu Jul 21 22:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:28:04.573266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde2f94-08af-4d16-871b-17b955d7f674 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:28:04.574828 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5dde2f94-08af-4d16-871b-17b955d7f674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:28:04.683202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde2f94-08af-4d16-871b-17b955d7f674 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:28:04.759135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d67b15-b042-476f-9cf6-875f1d769e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:28:04.759830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d67b15-b042-476f-9cf6-875f1d769e92 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:28:04.760076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d67b15-b042-476f-9cf6-875f1d769e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:28:04.760356 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d67b15-b042-476f-9cf6-875f1d769e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:28:04.762085 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a5d67b15-b042-476f-9cf6-875f1d769e92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:28:04.772036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:28:04.873479 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d67b15-b042-476f-9cf6-875f1d769e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:28:04.874914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d67b15-b042-476f-9cf6-875f1d769e92 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:28:04.894136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 746/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:28:14.530967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=82 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:28:14.891608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=50 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:45.294599 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b75f9-01a9-4ad8-adc4-b24b6a9c7cb1 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:45.295235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b75f9-01a9-4ad8-adc4-b24b6a9c7cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:45.295235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b75f9-01a9-4ad8-adc4-b24b6a9c7cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:45.368103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-129b75f9-01a9-4ad8-adc4-b24b6a9c7cb1 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:29:45.369338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-129b75f9-01a9-4ad8-adc4-b24b6a9c7cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:45.469634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 748/1494] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:29:45.477936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b3293-4b36-4029-86ec-30bf1d073637 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:45.478438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b3293-4b36-4029-86ec-30bf1d073637 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:45.478546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b3293-4b36-4029-86ec-30bf1d073637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:45.554600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-4f4b3293-4b36-4029-86ec-30bf1d073637 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:29:45.555738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4f4b3293-4b36-4029-86ec-30bf1d073637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:45.654126 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 747/1495] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:29:45.658749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b75f9-01a9-4ad8-adc4-b24b6a9c7cb1 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:45.660549 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-129b75f9-01a9-4ad8-adc4-b24b6a9c7cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:45.672111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b75f9-01a9-4ad8-adc4-b24b6a9c7cb1 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:45.733448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:45.767879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b86f4-e114-4c42-b355-7be534407447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:45.769660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b86f4-e114-4c42-b355-7be534407447 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:45.769945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b86f4-e114-4c42-b355-7be534407447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:45.770228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b86f4-e114-4c42-b355-7be534407447 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:45.770960 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b86f4-e114-4c42-b355-7be534407447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:29:45.771274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b86f4-e114-4c42-b355-7be534407447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:45.772295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-127b86f4-e114-4c42-b355-7be534407447 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:45.772295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: warnings.warn(msg) Jul 21 22:29:45.772295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: Jul 21 22:29:45.773231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b86f4-e114-4c42-b355-7be534407447 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:45.784961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 749/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:29:45.793388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b3293-4b36-4029-86ec-30bf1d073637 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:45.804170 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b3293-4b36-4029-86ec-30bf1d073637 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:45.866145 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:45.901078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28420e9b-ef8b-4d8e-909b-e47fa29b50ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:45.901746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28420e9b-ef8b-4d8e-909b-e47fa29b50ba admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:45.902032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28420e9b-ef8b-4d8e-909b-e47fa29b50ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:45.902314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28420e9b-ef8b-4d8e-909b-e47fa29b50ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:45.903005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28420e9b-ef8b-4d8e-909b-e47fa29b50ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:45.904839 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: WARNING py.warnings [None req-28420e9b-ef8b-4d8e-909b-e47fa29b50ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:45.904839 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: warnings.warn(msg) Jul 21 22:29:45.904839 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: Jul 21 22:29:45.906052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28420e9b-ef8b-4d8e-909b-e47fa29b50ba admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:45.961382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 748/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:29:45 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:29:45.965559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b86f4-e114-4c42-b355-7be534407447 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:45.967094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-127b86f4-e114-4c42-b355-7be534407447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:46.000127 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b86f4-e114-4c42-b355-7be534407447 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:46.094023 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f541c4-6e05-486b-a78e-73b9bdc1bbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:46.094687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f541c4-6e05-486b-a78e-73b9bdc1bbaf admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:46.094965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f541c4-6e05-486b-a78e-73b9bdc1bbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:46.095237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f541c4-6e05-486b-a78e-73b9bdc1bbaf admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:46.095958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f541c4-6e05-486b-a78e-73b9bdc1bbaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:46.097055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-33f541c4-6e05-486b-a78e-73b9bdc1bbaf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:46.097055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: warnings.warn(msg) Jul 21 22:29:46.097055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: Jul 21 22:29:46.098246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f541c4-6e05-486b-a78e-73b9bdc1bbaf admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:46.139405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 750/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:29:45 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:29:46.143591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28420e9b-ef8b-4d8e-909b-e47fa29b50ba admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:46.145173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-28420e9b-ef8b-4d8e-909b-e47fa29b50ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:46.211334 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28420e9b-ef8b-4d8e-909b-e47fa29b50ba admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:46.301618 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:46.314102 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817af01c-386f-4452-9c5a-f005121ae1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:46.314776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817af01c-386f-4452-9c5a-f005121ae1af admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:46.315059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817af01c-386f-4452-9c5a-f005121ae1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:46.315443 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817af01c-386f-4452-9c5a-f005121ae1af admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:46.316179 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817af01c-386f-4452-9c5a-f005121ae1af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:46.317265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: WARNING py.warnings [None req-817af01c-386f-4452-9c5a-f005121ae1af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:46.317265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: warnings.warn(msg) Jul 21 22:29:46.317265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: Jul 21 22:29:46.318144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817af01c-386f-4452-9c5a-f005121ae1af admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:46.330549 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 749/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:29:46.334683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f541c4-6e05-486b-a78e-73b9bdc1bbaf admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:46.336232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-33f541c4-6e05-486b-a78e-73b9bdc1bbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:46.348712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f541c4-6e05-486b-a78e-73b9bdc1bbaf admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:46.374462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:46.447862 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9d000-2e6c-4b25-b0a9-577a0cec5249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:46.448634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9d000-2e6c-4b25-b0a9-577a0cec5249 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:46.448846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9d000-2e6c-4b25-b0a9-577a0cec5249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:46.449115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9d000-2e6c-4b25-b0a9-577a0cec5249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:46.449836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9d000-2e6c-4b25-b0a9-577a0cec5249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:46.450854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-7ef9d000-2e6c-4b25-b0a9-577a0cec5249 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:46.450854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: warnings.warn(msg) Jul 21 22:29:46.450854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: Jul 21 22:29:46.451735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9d000-2e6c-4b25-b0a9-577a0cec5249 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:46.464333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 751/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:29:46.468693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817af01c-386f-4452-9c5a-f005121ae1af admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:46.470244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-817af01c-386f-4452-9c5a-f005121ae1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:46.504410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817af01c-386f-4452-9c5a-f005121ae1af admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:46.599208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54aca20-454a-4db6-b9e9-c4e93cefcc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:46.600109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aca20-454a-4db6-b9e9-c4e93cefcc28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:46.600395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aca20-454a-4db6-b9e9-c4e93cefcc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:46.600727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54aca20-454a-4db6-b9e9-c4e93cefcc28 admin admin] PATH_INFO: `/v3/projects/a7e54997d4bb4e738960f60847ef4509/users/7da5b34033b84d1dbdd9c7324cf2a9b1/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:46.623318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aca20-454a-4db6-b9e9-c4e93cefcc28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e54997d4bb4e738960f60847ef4509, user_id=7da5b34033b84d1dbdd9c7324cf2a9b1, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:46.624368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: WARNING py.warnings [None req-f54aca20-454a-4db6-b9e9-c4e93cefcc28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:46.624368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: warnings.warn(msg) Jul 21 22:29:46.624368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: Jul 21 22:29:46.626345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54aca20-454a-4db6-b9e9-c4e93cefcc28 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:46.651706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 750/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:29:46 2022] PUT /identity/v3/projects/a7e54997d4bb4e738960f60847ef4509/users/7da5b34033b84d1dbdd9c7324cf2a9b1/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:29:46.664539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b115aed-bf1c-42d0-9ecd-20fd3ad850e4 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:46.664840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b115aed-bf1c-42d0-9ecd-20fd3ad850e4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:46.665159 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b115aed-bf1c-42d0-9ecd-20fd3ad850e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:46.725837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3b115aed-bf1c-42d0-9ecd-20fd3ad850e4 None None] MFA Rules not processed for user `7da5b34033b84d1dbdd9c7324cf2a9b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:29:46.726981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3b115aed-bf1c-42d0-9ecd-20fd3ad850e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:46.803008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 752/1502] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:29:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:29:46.816331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fd323-8e89-49a2-961e-e4cb04bdb7df None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:46.816659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fd323-8e89-49a2-961e-e4cb04bdb7df None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:46.816946 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fd323-8e89-49a2-961e-e4cb04bdb7df None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:46.818344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 751/1503] 10.222.0.42 () {54 vars in 964 bytes} [Thu Jul 21 22:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:29:46.823419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614999b2-4621-45af-b56c-d43725982884 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:46.823715 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614999b2-4621-45af-b56c-d43725982884 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:46.823998 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614999b2-4621-45af-b56c-d43725982884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:46.857167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:46.902066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-614999b2-4621-45af-b56c-d43725982884 None None] MFA Rules not processed for user `a697d1e7e42048a8a1fb997bcc7b3fab`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:29:46.903276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-614999b2-4621-45af-b56c-d43725982884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:46.979601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 753/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Thu Jul 21 22:29:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:29:46.987820 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fd323-8e89-49a2-961e-e4cb04bdb7df None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:46.989343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0a3fd323-8e89-49a2-961e-e4cb04bdb7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:47.039826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:47.096435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fd323-8e89-49a2-961e-e4cb04bdb7df None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:47.171748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d43746-bac5-45c4-a154-e49109918ac8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:47.172400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d43746-bac5-45c4-a154-e49109918ac8 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:47.172723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d43746-bac5-45c4-a154-e49109918ac8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:47.173002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d43746-bac5-45c4-a154-e49109918ac8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:47.174753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b6d43746-bac5-45c4-a154-e49109918ac8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:47.400976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d43746-bac5-45c4-a154-e49109918ac8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:47.402400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d43746-bac5-45c4-a154-e49109918ac8 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:47.405890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:47.421742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 752/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:29:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:29:47.464539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525855be-b97a-445d-90ae-872f72c2efe7 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:47.464960 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525855be-b97a-445d-90ae-872f72c2efe7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:47.465099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525855be-b97a-445d-90ae-872f72c2efe7 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:47.466386 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 754/1506] 10.222.0.42 () {52 vars in 829 bytes} [Thu Jul 21 22:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:29:47.469903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d43746-bac5-45c4-a154-e49109918ac8 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:47.471103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b6d43746-bac5-45c4-a154-e49109918ac8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:47.570074 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d43746-bac5-45c4-a154-e49109918ac8 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:47.665003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:47.665794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:47.666082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:47.666361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] PATH_INFO: `/v3/users/7da5b34033b84d1dbdd9c7324cf2a9b1` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:47.680092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da5b34033b84d1dbdd9c7324cf2a9b1)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:47.681171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: WARNING py.warnings [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:47.681171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: warnings.warn(msg) Jul 21 22:29:47.681171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: Jul 21 22:29:47.682376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:47.692908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:47.747177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da5b34033b84d1dbdd9c7324cf2a9b1'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:47.756702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da5b34033b84d1dbdd9c7324cf2a9b1'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:47.757138 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da5b34033b84d1dbdd9c7324cf2a9b1'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:47.769742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da5b34033b84d1dbdd9c7324cf2a9b1'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:47.770342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da5b34033b84d1dbdd9c7324cf2a9b1'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:47.796071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 753/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:29:47 2022] DELETE /identity/v3/users/7da5b34033b84d1dbdd9c7324cf2a9b1 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:29:47.801403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe4be0-7be0-402a-b08d-c490c2db365e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:47.801697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe4be0-7be0-402a-b08d-c490c2db365e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:47.802006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe4be0-7be0-402a-b08d-c490c2db365e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:47.856941 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d3fe4be0-7be0-402a-b08d-c490c2db365e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:29:47.858148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d3fe4be0-7be0-402a-b08d-c490c2db365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:47.933962 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:48.066743 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 755/1508] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:29:48.076399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:48.077656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:48.178178 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5da687-6e3b-4849-97a4-5275d71abf7c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:48.241060 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:48.261765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32050cff-3bcb-49d1-8627-15077188fa1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:48.262418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32050cff-3bcb-49d1-8627-15077188fa1f service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:48.262704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32050cff-3bcb-49d1-8627-15077188fa1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:48.262982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32050cff-3bcb-49d1-8627-15077188fa1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:48.264942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-32050cff-3bcb-49d1-8627-15077188fa1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:48.372913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32050cff-3bcb-49d1-8627-15077188fa1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:48.374358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32050cff-3bcb-49d1-8627-15077188fa1f service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:48.392316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 754/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:29:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:29:48.750555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe4be0-7be0-402a-b08d-c490c2db365e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:29:48.751802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d3fe4be0-7be0-402a-b08d-c490c2db365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:29:48.764256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe4be0-7be0-402a-b08d-c490c2db365e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:29:48.876512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:29:48.877382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:29:48.878286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:29:48.878567 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] PATH_INFO: `/v3/projects/a7e54997d4bb4e738960f60847ef4509` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:29:48.889044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7e54997d4bb4e738960f60847ef4509)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:29:48.890089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:29:48.890089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: warnings.warn(msg) Jul 21 22:29:48.890089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: Jul 21 22:29:48.891336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:29:48.951763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7e54997d4bb4e738960f60847ef4509', 'request_id': 'req-7bef2369-d1a9-4530-abf3-6857eec20731'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:48.957943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7e54997d4bb4e738960f60847ef4509', 'request_id': 'req-7bef2369-d1a9-4530-abf3-6857eec20731'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:48.967834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7e54997d4bb4e738960f60847ef4509', 'request_id': 'req-7bef2369-d1a9-4530-abf3-6857eec20731'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:48.997584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] The token cache is being invalidate because project a7e54997d4bb4e738960f60847ef4509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:29:48.997869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:48.998298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7bef2369-d1a9-4530-abf3-6857eec20731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:29:48.999755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 756/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:29:48 2022] DELETE /identity/v3/projects/a7e54997d4bb4e738960f60847ef4509 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:29:58.390016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=62 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:29:58.995558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:31:34.656769 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87028a1-fd5a-486c-b1d1-2ce9bdce4d6d None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:34.657384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87028a1-fd5a-486c-b1d1-2ce9bdce4d6d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:34.657384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87028a1-fd5a-486c-b1d1-2ce9bdce4d6d None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:34.658676 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:31:34.664337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728b23d-bb10-4c70-82f8-dff940d97a64 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:34.664841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728b23d-bb10-4c70-82f8-dff940d97a64 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:34.664948 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728b23d-bb10-4c70-82f8-dff940d97a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:34.750541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4728b23d-bb10-4c70-82f8-dff940d97a64 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:34.751686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4728b23d-bb10-4c70-82f8-dff940d97a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:34.855013 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:31:35.008981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87028a1-fd5a-486c-b1d1-2ce9bdce4d6d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:31:35.011281 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c87028a1-fd5a-486c-b1d1-2ce9bdce4d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:35.131573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87028a1-fd5a-486c-b1d1-2ce9bdce4d6d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:31:35.137407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f864096-4b9d-4e15-b470-4978685196a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:31:35.138054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f864096-4b9d-4e15-b470-4978685196a1 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:35.138335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f864096-4b9d-4e15-b470-4978685196a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:35.138604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f864096-4b9d-4e15-b470-4978685196a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:35.147321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f864096-4b9d-4e15-b470-4978685196a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:31:35.148918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f864096-4b9d-4e15-b470-4978685196a1 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:31:35.158100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 756/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:31:35.999853 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29888dc-567b-49f8-ba4e-26eae5cd8997 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:36.000401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29888dc-567b-49f8-ba4e-26eae5cd8997 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:36.000401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29888dc-567b-49f8-ba4e-26eae5cd8997 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:36.001938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:31:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:31:36.007371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51763f1d-36e1-419f-9255-04aa51af41b9 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:36.007831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51763f1d-36e1-419f-9255-04aa51af41b9 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:36.007937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51763f1d-36e1-419f-9255-04aa51af41b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:36.026404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-51763f1d-36e1-419f-9255-04aa51af41b9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:36.027599 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-51763f1d-36e1-419f-9255-04aa51af41b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:36.036004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:31:36.190996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29888dc-567b-49f8-ba4e-26eae5cd8997 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:31:36.206604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29888dc-567b-49f8-ba4e-26eae5cd8997 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:31:36.212760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c724b-17e1-4b98-a54c-90723e7ec6cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:31:36.213403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c724b-17e1-4b98-a54c-90723e7ec6cc service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:36.213687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c724b-17e1-4b98-a54c-90723e7ec6cc service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:36.213966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c724b-17e1-4b98-a54c-90723e7ec6cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:36.222311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c724b-17e1-4b98-a54c-90723e7ec6cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:31:36.224383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c724b-17e1-4b98-a54c-90723e7ec6cc service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:31:36.234916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 759/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:31:36.310715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57635d-d494-4a11-a0be-4fdd0589bffd None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:36.311013 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57635d-d494-4a11-a0be-4fdd0589bffd None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:36.311328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57635d-d494-4a11-a0be-4fdd0589bffd None None] PATH_INFO: `/v3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:36.312771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 758/1517] 10.222.0.40 () {54 vars in 925 bytes} [Thu Jul 21 22:31:36 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:31:36.320283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa15242-ed32-47ce-8cd1-bd1790737959 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:36.320697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa15242-ed32-47ce-8cd1-bd1790737959 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:36.321043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa15242-ed32-47ce-8cd1-bd1790737959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:36.380377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-dfa15242-ed32-47ce-8cd1-bd1790737959 None None] MFA Rules not processed for user `705ff0e73e95499e873b1df2a24fefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:36.382290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dfa15242-ed32-47ce-8cd1-bd1790737959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:36.399508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:31:36.455592 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 760/1518] 10.222.0.40 () {58 vars in 1015 bytes} [Thu Jul 21 22:31:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:31:46.021597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=56 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:31:46.452685 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=23 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:31:46.954712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb7aa1-c715-4d31-82c2-6cd354b080aa None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:46.954934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb7aa1-c715-4d31-82c2-6cd354b080aa None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:46.955225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb7aa1-c715-4d31-82c2-6cd354b080aa None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:46.956673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 759/1519] 10.222.0.42 () {54 vars in 961 bytes} [Thu Jul 21 22:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:31:46.962562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a61149-951c-4415-9ee0-0bb0c07769e3 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:46.962925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a61149-951c-4415-9ee0-0bb0c07769e3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:46.963274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a61149-951c-4415-9ee0-0bb0c07769e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:46.981856 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-56a61149-951c-4415-9ee0-0bb0c07769e3 None None] MFA Rules not processed for user `705ff0e73e95499e873b1df2a24fefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:46.983168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-56a61149-951c-4415-9ee0-0bb0c07769e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:46.991996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 761/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Jul 21 22:31:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:31:46.999344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb7aa1-c715-4d31-82c2-6cd354b080aa None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:31:47.013719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb7aa1-c715-4d31-82c2-6cd354b080aa None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:31:47.019077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e57b85-3587-4939-a232-6ec35684df21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:31:47.019717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e57b85-3587-4939-a232-6ec35684df21 service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:47.020008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e57b85-3587-4939-a232-6ec35684df21 service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:47.021023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e57b85-3587-4939-a232-6ec35684df21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:47.029459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e57b85-3587-4939-a232-6ec35684df21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:31:47.031045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e57b85-3587-4939-a232-6ec35684df21 service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:31:47.040876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 760/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:31:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:31:47.504567 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae86927-c7c5-4a5f-ac50-e5bef18724b3 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:47.505120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae86927-c7c5-4a5f-ac50-e5bef18724b3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:47.505120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae86927-c7c5-4a5f-ac50-e5bef18724b3 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:47.506505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:31:47.511560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafc23e-1242-40da-b2d6-d98700e684dd None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:47.512110 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafc23e-1242-40da-b2d6-d98700e684dd None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:47.512391 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafc23e-1242-40da-b2d6-d98700e684dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:47.531127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-3fafc23e-1242-40da-b2d6-d98700e684dd None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:47.532304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3fafc23e-1242-40da-b2d6-d98700e684dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:47.540874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:31:47.701294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae86927-c7c5-4a5f-ac50-e5bef18724b3 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:31:47.716787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae86927-c7c5-4a5f-ac50-e5bef18724b3 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:31:47.722241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3d946-5641-495a-953f-8dfc7ad4577a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:31:47.722881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3d946-5641-495a-953f-8dfc7ad4577a service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:47.723164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3d946-5641-495a-953f-8dfc7ad4577a service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:47.723443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3d946-5641-495a-953f-8dfc7ad4577a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:47.732734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3d946-5641-495a-953f-8dfc7ad4577a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:31:47.734320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3d946-5641-495a-953f-8dfc7ad4577a service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:31:47.744163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 763/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:31:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:31:48.607861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb503383-2f43-44d7-a180-176c9bef2a70 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:48.608369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb503383-2f43-44d7-a180-176c9bef2a70 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:48.608499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb503383-2f43-44d7-a180-176c9bef2a70 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:48.609873 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:31:48.615187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e14c3-a360-4042-8af0-39eeae36f539 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:48.615629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e14c3-a360-4042-8af0-39eeae36f539 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:48.615778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e14c3-a360-4042-8af0-39eeae36f539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:48.634189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8a7e14c3-a360-4042-8af0-39eeae36f539 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:48.635406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8a7e14c3-a360-4042-8af0-39eeae36f539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:48.643683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:31:48.800107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb503383-2f43-44d7-a180-176c9bef2a70 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:31:48.815217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb503383-2f43-44d7-a180-176c9bef2a70 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:31:48.820908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9dc9b-b9a5-48aa-ab09-66feead7540d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:31:48.821558 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9dc9b-b9a5-48aa-ab09-66feead7540d service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:48.821847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9dc9b-b9a5-48aa-ab09-66feead7540d service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:48.822131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9dc9b-b9a5-48aa-ab09-66feead7540d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:48.830861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9dc9b-b9a5-48aa-ab09-66feead7540d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:31:48.832452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9dc9b-b9a5-48aa-ab09-66feead7540d service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:31:48.843034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 763/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:31:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:31:53.095045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f0764-c47c-4c32-b79f-401b1c41523f None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:53.095620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f0764-c47c-4c32-b79f-401b1c41523f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:53.095620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f0764-c47c-4c32-b79f-401b1c41523f None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:53.097074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:31:53.103526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aae837-8015-464e-afc4-7c79d20a37eb None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:53.103977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aae837-8015-464e-afc4-7c79d20a37eb None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:53.104080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aae837-8015-464e-afc4-7c79d20a37eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:53.123917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-65aae837-8015-464e-afc4-7c79d20a37eb None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:53.125156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-65aae837-8015-464e-afc4-7c79d20a37eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:53.133678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:31:53.211046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f0764-c47c-4c32-b79f-401b1c41523f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:31:53.212839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-887f0764-c47c-4c32-b79f-401b1c41523f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:53.294699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f0764-c47c-4c32-b79f-401b1c41523f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:31:53.298352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304025f-c8ce-469d-af8c-058d8321a93e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:31:53.299000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304025f-c8ce-469d-af8c-058d8321a93e service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:53.299285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304025f-c8ce-469d-af8c-058d8321a93e service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:53.299560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304025f-c8ce-469d-af8c-058d8321a93e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:53.309981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304025f-c8ce-469d-af8c-058d8321a93e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:31:53.311431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304025f-c8ce-469d-af8c-058d8321a93e service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:31:53.322157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 766/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:31:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:31:55.382345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8a29d-309e-49f9-8ceb-19d0d4f26db5 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:55.382931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8a29d-309e-49f9-8ceb-19d0d4f26db5 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:55.382931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8a29d-309e-49f9-8ceb-19d0d4f26db5 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:55.384276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:31:55.389755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ee656-aa7c-477f-a05e-6eb153c9ca0e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:55.390202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ee656-aa7c-477f-a05e-6eb153c9ca0e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:55.390310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ee656-aa7c-477f-a05e-6eb153c9ca0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:55.412666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-303ee656-aa7c-477f-a05e-6eb153c9ca0e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:55.413894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-303ee656-aa7c-477f-a05e-6eb153c9ca0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:55.421807 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:31:55.459465 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8a29d-309e-49f9-8ceb-19d0d4f26db5 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:31:55.470037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8a29d-309e-49f9-8ceb-19d0d4f26db5 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:31:55.473649 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaac552-4c8d-490e-94c7-fc689a1e6f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:31:55.474292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaac552-4c8d-490e-94c7-fc689a1e6f96 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:55.474576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaac552-4c8d-490e-94c7-fc689a1e6f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:55.474853 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaac552-4c8d-490e-94c7-fc689a1e6f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:55.483639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaac552-4c8d-490e-94c7-fc689a1e6f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:31:55.485182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaac552-4c8d-490e-94c7-fc689a1e6f96 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:31:55.494755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 766/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:31:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:31:59.295136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f5a09-a992-45b1-8d9c-f8c0eced3a60 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:59.295842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f5a09-a992-45b1-8d9c-f8c0eced3a60 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:59.295842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f5a09-a992-45b1-8d9c-f8c0eced3a60 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:59.297091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:31:59.302430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28343b60-37e0-4070-8340-6b6ea3ff1010 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:59.302925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28343b60-37e0-4070-8340-6b6ea3ff1010 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:59.303030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28343b60-37e0-4070-8340-6b6ea3ff1010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:59.321806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-28343b60-37e0-4070-8340-6b6ea3ff1010 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:31:59.323022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-28343b60-37e0-4070-8340-6b6ea3ff1010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:59.331051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:31:59.649303 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f5a09-a992-45b1-8d9c-f8c0eced3a60 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:31:59.651095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5a3f5a09-a992-45b1-8d9c-f8c0eced3a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:31:59.736157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f5a09-a992-45b1-8d9c-f8c0eced3a60 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:31:59.740530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcaeafd-d35f-4adb-925a-ce4ac93f2dc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:31:59.741196 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcaeafd-d35f-4adb-925a-ce4ac93f2dc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:31:59.741481 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcaeafd-d35f-4adb-925a-ce4ac93f2dc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:31:59.741761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcaeafd-d35f-4adb-925a-ce4ac93f2dc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:31:59.751648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcaeafd-d35f-4adb-925a-ce4ac93f2dc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:31:59.753147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcaeafd-d35f-4adb-925a-ce4ac93f2dc4 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:31:59.763540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 769/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:01.926494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060968c-42db-4ff4-8be8-59e2b1951716 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:01.927391 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060968c-42db-4ff4-8be8-59e2b1951716 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:01.927679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060968c-42db-4ff4-8be8-59e2b1951716 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:01.929100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:32:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:32:01.934806 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60184d5-8bf8-49b4-9e16-d78f4b73269b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:01.935307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60184d5-8bf8-49b4-9e16-d78f4b73269b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:01.935412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60184d5-8bf8-49b4-9e16-d78f4b73269b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:01.955303 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f60184d5-8bf8-49b4-9e16-d78f4b73269b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:32:01.956587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f60184d5-8bf8-49b4-9e16-d78f4b73269b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:01.964757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:32:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:32:02.289171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060968c-42db-4ff4-8be8-59e2b1951716 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:32:02.291509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1060968c-42db-4ff4-8be8-59e2b1951716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:32:02.306083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060968c-42db-4ff4-8be8-59e2b1951716 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:32:02.310354 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334f964-f22b-4cca-ae22-01590ad342df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:32:02.311209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f964-f22b-4cca-ae22-01590ad342df service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:32:02.311592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f964-f22b-4cca-ae22-01590ad342df service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:32:02.311968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f964-f22b-4cca-ae22-01590ad342df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:32:02.322826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334f964-f22b-4cca-ae22-01590ad342df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:32:02.324259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334f964-f22b-4cca-ae22-01590ad342df service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:32:02.335433 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 769/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:32:11.949264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=88 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:32:12.329919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=33 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:40.081155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edeaae-b7d0-4ad5-aaa3-47195529045d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.082133 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edeaae-b7d0-4ad5-aaa3-47195529045d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.082133 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edeaae-b7d0-4ad5-aaa3-47195529045d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.109799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-71edeaae-b7d0-4ad5-aaa3-47195529045d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:40.111054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-71edeaae-b7d0-4ad5-aaa3-47195529045d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.130188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 771/1540] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:40.141763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51fcf8-3a92-4393-8cef-179f91acaf88 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.143556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51fcf8-3a92-4393-8cef-179f91acaf88 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.144436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51fcf8-3a92-4393-8cef-179f91acaf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.181868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-aa51fcf8-3a92-4393-8cef-179f91acaf88 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:40.183364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-aa51fcf8-3a92-4393-8cef-179f91acaf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.193460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 770/1541] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:40.198237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edeaae-b7d0-4ad5-aaa3-47195529045d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.219097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f2446-1662-497c-ae55-f2e8ab586fcb None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.219707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f2446-1662-497c-ae55-f2e8ab586fcb None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.220201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f2446-1662-497c-ae55-f2e8ab586fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.224221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edeaae-b7d0-4ad5-aaa3-47195529045d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.242495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f40a74-ecaf-4598-b1c0-1244054244a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.243356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f40a74-ecaf-4598-b1c0-1244054244a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.243816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f40a74-ecaf-4598-b1c0-1244054244a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.244102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f40a74-ecaf-4598-b1c0-1244054244a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.248509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f40a74-ecaf-4598-b1c0-1244054244a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:40.248509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f40a74-ecaf-4598-b1c0-1244054244a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.248509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f40a74-ecaf-4598-b1c0-1244054244a5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.258135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-477f2446-1662-497c-ae55-f2e8ab586fcb None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:40.260750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-477f2446-1662-497c-ae55-f2e8ab586fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.275639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 771/1542] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:40.279601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 772/1543] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:40.286065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b52d4b-cffd-4658-ab4f-666c76304804 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.286749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b52d4b-cffd-4658-ab4f-666c76304804 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.287339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b52d4b-cffd-4658-ab4f-666c76304804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.288359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f40a74-ecaf-4598-b1c0-1244054244a5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.303903 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f40a74-ecaf-4598-b1c0-1244054244a5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.317344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-d5b52d4b-cffd-4658-ab4f-666c76304804 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:40.320607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d5b52d4b-cffd-4658-ab4f-666c76304804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.322529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643d219-539c-4b3d-947e-584aae7dc018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.323557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643d219-539c-4b3d-947e-584aae7dc018 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.323842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643d219-539c-4b3d-947e-584aae7dc018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.324120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643d219-539c-4b3d-947e-584aae7dc018 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.324880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643d219-539c-4b3d-947e-584aae7dc018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.326151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-4643d219-539c-4b3d-947e-584aae7dc018 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:33:40.326151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: warnings.warn(msg) Jul 21 22:33:40.326151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: Jul 21 22:33:40.328099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643d219-539c-4b3d-947e-584aae7dc018 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.338399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 772/1544] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:40.343743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b52d4b-cffd-4658-ab4f-666c76304804 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.355769 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b52d4b-cffd-4658-ab4f-666c76304804 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.373383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6686f-67d3-4b57-ac26-278c25d18f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.374908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6686f-67d3-4b57-ac26-278c25d18f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.375230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6686f-67d3-4b57-ac26-278c25d18f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.375708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6686f-67d3-4b57-ac26-278c25d18f3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.376523 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6686f-67d3-4b57-ac26-278c25d18f3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:40.377012 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6686f-67d3-4b57-ac26-278c25d18f3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.380753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: WARNING py.warnings [None req-e5d6686f-67d3-4b57-ac26-278c25d18f3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:33:40.380753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: warnings.warn(msg) Jul 21 22:33:40.380753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: Jul 21 22:33:40.381943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6686f-67d3-4b57-ac26-278c25d18f3a admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.406854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 773/1545] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:33:40.408294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed4a2d-c594-454f-9a41-69d3253be8b0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.408617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed4a2d-c594-454f-9a41-69d3253be8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.409197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 773/1546] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:33:40.409470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed4a2d-c594-454f-9a41-69d3253be8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.421264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6686f-67d3-4b57-ac26-278c25d18f3a admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.445329 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6686f-67d3-4b57-ac26-278c25d18f3a admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.458666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f4ed4a2d-c594-454f-9a41-69d3253be8b0 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:40.460857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f4ed4a2d-c594-454f-9a41-69d3253be8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.580262 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330bfc1-1622-4ccc-b907-f9ced75883cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.581491 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330bfc1-1622-4ccc-b907-f9ced75883cd admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.582024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330bfc1-1622-4ccc-b907-f9ced75883cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.582548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330bfc1-1622-4ccc-b907-f9ced75883cd admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.592657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330bfc1-1622-4ccc-b907-f9ced75883cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.594949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: WARNING py.warnings [None req-d330bfc1-1622-4ccc-b907-f9ced75883cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:33:40.594949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: warnings.warn(msg) Jul 21 22:33:40.594949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: Jul 21 22:33:40.596923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330bfc1-1622-4ccc-b907-f9ced75883cd admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.616508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 774/1547] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:40.616508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed4a2d-c594-454f-9a41-69d3253be8b0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.626709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed4a2d-c594-454f-9a41-69d3253be8b0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.645212 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca81de3-46cf-4488-9857-4889affcd8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.647056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca81de3-46cf-4488-9857-4889affcd8ee admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.647367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca81de3-46cf-4488-9857-4889affcd8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.647644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca81de3-46cf-4488-9857-4889affcd8ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.651265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca81de3-46cf-4488-9857-4889affcd8ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.656864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 774/1548] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/users => generated 290 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:40.659000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca81de3-46cf-4488-9857-4889affcd8ee admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.660978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63956f14-1a71-484b-bf94-44ffad26c2c8 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.661271 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63956f14-1a71-484b-bf94-44ffad26c2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.661550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63956f14-1a71-484b-bf94-44ffad26c2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.703434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-63956f14-1a71-484b-bf94-44ffad26c2c8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:40.704883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-63956f14-1a71-484b-bf94-44ffad26c2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.720083 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 775/1549] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:40.721662 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dd517-7ea8-4901-9795-47436b51d5d2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.721959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dd517-7ea8-4901-9795-47436b51d5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.722200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dd517-7ea8-4901-9795-47436b51d5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.725568 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 775/1550] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:40.726997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a001662-1767-4947-bb6d-706b34110218 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.727473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a001662-1767-4947-bb6d-706b34110218 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.727655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a001662-1767-4947-bb6d-706b34110218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.751037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-4a001662-1767-4947-bb6d-706b34110218 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:40.752429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4a001662-1767-4947-bb6d-706b34110218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.759159 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a03dd517-7ea8-4901-9795-47436b51d5d2 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:40.762131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a03dd517-7ea8-4901-9795-47436b51d5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.831897 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 776/1551] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:40.832945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a001662-1767-4947-bb6d-706b34110218 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.845864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a001662-1767-4947-bb6d-706b34110218 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.851074 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa4f5a-e118-41d7-8e85-edf2f6750519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.851775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa4f5a-e118-41d7-8e85-edf2f6750519 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.852060 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa4f5a-e118-41d7-8e85-edf2f6750519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.852339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa4f5a-e118-41d7-8e85-edf2f6750519 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.853297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa4f5a-e118-41d7-8e85-edf2f6750519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.855014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa4f5a-e118-41d7-8e85-edf2f6750519 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.871877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 777/1552] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:40.874251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc8732-2371-4fc1-bbd4-d90324c6ed37 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.874612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc8732-2371-4fc1-bbd4-d90324c6ed37 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.875093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc8732-2371-4fc1-bbd4-d90324c6ed37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.896549 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-79dc8732-2371-4fc1-bbd4-d90324c6ed37 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:40.898054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-79dc8732-2371-4fc1-bbd4-d90324c6ed37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.905675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 776/1553] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:40.908071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dd517-7ea8-4901-9795-47436b51d5d2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.909045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 778/1554] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:40.910765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5786c4e-b4e1-426c-bcd7-ea45ffa41270 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.911183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5786c4e-b4e1-426c-bcd7-ea45ffa41270 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.911467 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5786c4e-b4e1-426c-bcd7-ea45ffa41270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.931222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dd517-7ea8-4901-9795-47436b51d5d2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.933547 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f5786c4e-b4e1-426c-bcd7-ea45ffa41270 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:40.934977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f5786c4e-b4e1-426c-bcd7-ea45ffa41270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:40.942334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d527da-f2e3-4165-910c-6a1dbff39499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.944041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d527da-f2e3-4165-910c-6a1dbff39499 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.944937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d527da-f2e3-4165-910c-6a1dbff39499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.945426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 779/1555] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:40.951282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5786c4e-b4e1-426c-bcd7-ea45ffa41270 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.951528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d527da-f2e3-4165-910c-6a1dbff39499 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.951528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d527da-f2e3-4165-910c-6a1dbff39499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.951825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d527da-f2e3-4165-910c-6a1dbff39499 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.958506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5786c4e-b4e1-426c-bcd7-ea45ffa41270 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:40.964023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e780941-6440-4fa2-a79b-0a397390a8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:40.965228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e780941-6440-4fa2-a79b-0a397390a8cf admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.965228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e780941-6440-4fa2-a79b-0a397390a8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.965593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e780941-6440-4fa2-a79b-0a397390a8cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:40.966042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e780941-6440-4fa2-a79b-0a397390a8cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:40.966467 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e780941-6440-4fa2-a79b-0a397390a8cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:40.968987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e780941-6440-4fa2-a79b-0a397390a8cf admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:40.985874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 780/1556] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:40.985874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e780941-6440-4fa2-a79b-0a397390a8cf admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:40.986925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 777/1557] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/users => generated 295 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:40.988462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b8564-c7f7-4748-84fc-2d0d43e7e93d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:40.988880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b8564-c7f7-4748-84fc-2d0d43e7e93d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:40.989054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b8564-c7f7-4748-84fc-2d0d43e7e93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.001983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e780941-6440-4fa2-a79b-0a397390a8cf admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.007734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52faac0e-8ea8-4a9f-81a5-88e937a34e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.008463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faac0e-8ea8-4a9f-81a5-88e937a34e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.008785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faac0e-8ea8-4a9f-81a5-88e937a34e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.009053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faac0e-8ea8-4a9f-81a5-88e937a34e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.010088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52faac0e-8ea8-4a9f-81a5-88e937a34e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.010294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c42b8564-c7f7-4748-84fc-2d0d43e7e93d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:41.011473 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c42b8564-c7f7-4748-84fc-2d0d43e7e93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.011716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52faac0e-8ea8-4a9f-81a5-88e937a34e25 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.020203 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 778/1558] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:41.021573 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05975e5-df09-4be6-84e2-a4ae37448de7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.021853 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05975e5-df09-4be6-84e2-a4ae37448de7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.022125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05975e5-df09-4be6-84e2-a4ae37448de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.028901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 781/1559] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:41.030105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:41.031019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cd180-6398-4163-9eb3-5f17cff66ce3 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.031161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cd180-6398-4163-9eb3-5f17cff66ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.031378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cd180-6398-4163-9eb3-5f17cff66ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.040015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:41.048152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b05975e5-df09-4be6-84e2-a4ae37448de7 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:41.049559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b05975e5-df09-4be6-84e2-a4ae37448de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.058580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 779/1560] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:41.059450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05975e5-df09-4be6-84e2-a4ae37448de7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.060058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-655cd180-6398-4163-9eb3-5f17cff66ce3 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:41.063164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-655cd180-6398-4163-9eb3-5f17cff66ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.070916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05975e5-df09-4be6-84e2-a4ae37448de7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.075898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 782/1561] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:41.077105 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cd180-6398-4163-9eb3-5f17cff66ce3 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.079836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051daf82-1fdd-4f38-88d2-9bf674dd69f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.081061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051daf82-1fdd-4f38-88d2-9bf674dd69f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.082036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051daf82-1fdd-4f38-88d2-9bf674dd69f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.082980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051daf82-1fdd-4f38-88d2-9bf674dd69f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.084814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051daf82-1fdd-4f38-88d2-9bf674dd69f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:41.085374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051daf82-1fdd-4f38-88d2-9bf674dd69f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.088644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051daf82-1fdd-4f38-88d2-9bf674dd69f6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.089465 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cd180-6398-4163-9eb3-5f17cff66ce3 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.094615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3b0c6-e49b-4e98-88af-022ebb1e70e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.095468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3b0c6-e49b-4e98-88af-022ebb1e70e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.095976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3b0c6-e49b-4e98-88af-022ebb1e70e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.096327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3b0c6-e49b-4e98-88af-022ebb1e70e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.097301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3b0c6-e49b-4e98-88af-022ebb1e70e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.099086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3b0c6-e49b-4e98-88af-022ebb1e70e5 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.115656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 783/1562] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:41.115656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3b0c6-e49b-4e98-88af-022ebb1e70e5 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.116061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 780/1563] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:41.116188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce9087-eece-467c-9a9d-7e998e778df6 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.116705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce9087-eece-467c-9a9d-7e998e778df6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.117297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce9087-eece-467c-9a9d-7e998e778df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.131871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3b0c6-e49b-4e98-88af-022ebb1e70e5 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.138083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea396f1-6a52-400c-a2f3-ab1d1e8a01d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.138933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea396f1-6a52-400c-a2f3-ab1d1e8a01d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.139324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea396f1-6a52-400c-a2f3-ab1d1e8a01d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.140111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea396f1-6a52-400c-a2f3-ab1d1e8a01d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.141015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea396f1-6a52-400c-a2f3-ab1d1e8a01d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.143096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea396f1-6a52-400c-a2f3-ab1d1e8a01d0 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.152946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-dcce9087-eece-467c-9a9d-7e998e778df6 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:41.155252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dcce9087-eece-467c-9a9d-7e998e778df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.171302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 781/1564] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:41.173674 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce9087-eece-467c-9a9d-7e998e778df6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.181008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 784/1565] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:41.183155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea396f1-6a52-400c-a2f3-ab1d1e8a01d0 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.193399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce9087-eece-467c-9a9d-7e998e778df6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.267522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea396f1-6a52-400c-a2f3-ab1d1e8a01d0 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.273044 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16864b-efce-476b-9446-2c43d3bec6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.273979 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16864b-efce-476b-9446-2c43d3bec6b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.274268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16864b-efce-476b-9446-2c43d3bec6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.274556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16864b-efce-476b-9446-2c43d3bec6b5 admin admin] PATH_INFO: `/v3/projects/aa3208211a214b4a9c7face1a1fed6f0/users/c43bf6734c984c6e9d189660f903958b/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.290743 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e89c3b-6cf0-465e-ae9c-f8fe963a2738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.291921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e89c3b-6cf0-465e-ae9c-f8fe963a2738 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.292493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e89c3b-6cf0-465e-ae9c-f8fe963a2738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.292997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e89c3b-6cf0-465e-ae9c-f8fe963a2738 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.293189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16864b-efce-476b-9446-2c43d3bec6b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3208211a214b4a9c7face1a1fed6f0, user_id=c43bf6734c984c6e9d189660f903958b, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.294490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e89c3b-6cf0-465e-ae9c-f8fe963a2738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:41.295052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e89c3b-6cf0-465e-ae9c-f8fe963a2738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.297304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16864b-efce-476b-9446-2c43d3bec6b5 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.299572 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e89c3b-6cf0-465e-ae9c-f8fe963a2738 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.313867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 785/1566] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:41 2022] PUT /identity/v3/projects/aa3208211a214b4a9c7face1a1fed6f0/users/c43bf6734c984c6e9d189660f903958b/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:41.314781 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16864b-efce-476b-9446-2c43d3bec6b5 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.325750 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 782/1567] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:41.329564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607f5ea-93ea-400c-9e0a-8974ce457647 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.330082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607f5ea-93ea-400c-9e0a-8974ce457647 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.330590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607f5ea-93ea-400c-9e0a-8974ce457647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.367679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a607f5ea-93ea-400c-9e0a-8974ce457647 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:41.369777 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a607f5ea-93ea-400c-9e0a-8974ce457647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.388198 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16864b-efce-476b-9446-2c43d3bec6b5 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.393370 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434934d1-f440-4599-82f2-433d201e31e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.394067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434934d1-f440-4599-82f2-433d201e31e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.394364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434934d1-f440-4599-82f2-433d201e31e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.394815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434934d1-f440-4599-82f2-433d201e31e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.395671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434934d1-f440-4599-82f2-433d201e31e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:41.395870 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434934d1-f440-4599-82f2-433d201e31e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.397600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434934d1-f440-4599-82f2-433d201e31e4 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.420623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 786/1568] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:41.420623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434934d1-f440-4599-82f2-433d201e31e4 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.436697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434934d1-f440-4599-82f2-433d201e31e4 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.442856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270fb93d-4ce3-4912-bd8b-bd4a9d047ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.443804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fb93d-4ce3-4912-bd8b-bd4a9d047ecb admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.444204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fb93d-4ce3-4912-bd8b-bd4a9d047ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.444672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fb93d-4ce3-4912-bd8b-bd4a9d047ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.445624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270fb93d-4ce3-4912-bd8b-bd4a9d047ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.447550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270fb93d-4ce3-4912-bd8b-bd4a9d047ecb admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.459371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 787/1569] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:41.460232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270fb93d-4ce3-4912-bd8b-bd4a9d047ecb admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.471712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270fb93d-4ce3-4912-bd8b-bd4a9d047ecb admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.476303 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 783/1570] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:41.476741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df81ae-448b-4b9e-9988-8164e646fc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.477446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df81ae-448b-4b9e-9988-8164e646fc72 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.477734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df81ae-448b-4b9e-9988-8164e646fc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.478015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df81ae-448b-4b9e-9988-8164e646fc72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.478737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df81ae-448b-4b9e-9988-8164e646fc72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.480743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df81ae-448b-4b9e-9988-8164e646fc72 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.482669 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fa9d4-9620-4ba6-bf39-cf55febfcb7f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.484059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fa9d4-9620-4ba6-bf39-cf55febfcb7f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.484570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fa9d4-9620-4ba6-bf39-cf55febfcb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.513570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 788/1571] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:41.515202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df81ae-448b-4b9e-9988-8164e646fc72 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.525524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:41.525757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2d9fa9d4-9620-4ba6-bf39-cf55febfcb7f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:41.527559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2d9fa9d4-9620-4ba6-bf39-cf55febfcb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.602616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df81ae-448b-4b9e-9988-8164e646fc72 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.610214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc9d21-b257-4f2b-80e4-124ac0715853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.610953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc9d21-b257-4f2b-80e4-124ac0715853 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.611235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc9d21-b257-4f2b-80e4-124ac0715853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.611511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc9d21-b257-4f2b-80e4-124ac0715853 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.612369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc9d21-b257-4f2b-80e4-124ac0715853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.614406 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc9d21-b257-4f2b-80e4-124ac0715853 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.626592 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 784/1572] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:41.628706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3e698-3c75-4774-9fc6-e8298f0478dc None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.629221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3e698-3c75-4774-9fc6-e8298f0478dc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.629728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3e698-3c75-4774-9fc6-e8298f0478dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.650352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 789/1573] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:41.650352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc9d21-b257-4f2b-80e4-124ac0715853 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.653266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c2d3e698-3c75-4774-9fc6-e8298f0478dc None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:41.654580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c2d3e698-3c75-4774-9fc6-e8298f0478dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:41.665374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 785/1574] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:41.666226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3e698-3c75-4774-9fc6-e8298f0478dc None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.666831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc9d21-b257-4f2b-80e4-124ac0715853 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.678104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3e698-3c75-4774-9fc6-e8298f0478dc None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.679480 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699402c0-59ce-44d7-8ce4-0eee23c8fbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.680681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699402c0-59ce-44d7-8ce4-0eee23c8fbf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.681217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699402c0-59ce-44d7-8ce4-0eee23c8fbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.681759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699402c0-59ce-44d7-8ce4-0eee23c8fbf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.683148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699402c0-59ce-44d7-8ce4-0eee23c8fbf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.683286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e11f8-7f75-4750-976e-e4b441a97387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.683976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e11f8-7f75-4750-976e-e4b441a97387 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.684265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e11f8-7f75-4750-976e-e4b441a97387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.684584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e11f8-7f75-4750-976e-e4b441a97387 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.685286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e11f8-7f75-4750-976e-e4b441a97387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.687751 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e11f8-7f75-4750-976e-e4b441a97387 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.688413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699402c0-59ce-44d7-8ce4-0eee23c8fbf3 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.691419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:41.724239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 790/1575] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/projects => generated 403 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:41.725266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699402c0-59ce-44d7-8ce4-0eee23c8fbf3 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.730350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 786/1576] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:41.730350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e11f8-7f75-4750-976e-e4b441a97387 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.809274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699402c0-59ce-44d7-8ce4-0eee23c8fbf3 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.857209 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e11f8-7f75-4750-976e-e4b441a97387 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.865843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27df9af-1913-4c0b-bd83-ad6cafe6b38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.867028 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27df9af-1913-4c0b-bd83-ad6cafe6b38e admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.867506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27df9af-1913-4c0b-bd83-ad6cafe6b38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.868003 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27df9af-1913-4c0b-bd83-ad6cafe6b38e admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.869443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27df9af-1913-4c0b-bd83-ad6cafe6b38e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:41.870009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27df9af-1913-4c0b-bd83-ad6cafe6b38e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.872952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27df9af-1913-4c0b-bd83-ad6cafe6b38e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.876464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ff20b-63e7-4113-93da-cbfe5c68cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:41.877499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ff20b-63e7-4113-93da-cbfe5c68cd4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:41.877698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ff20b-63e7-4113-93da-cbfe5c68cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:41.878415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ff20b-63e7-4113-93da-cbfe5c68cd4a admin admin] PATH_INFO: `/v3/projects/613f21f4b5164e6087e734f9b447e961/users/da46b59a0cb5430589aeef1693b2ce50/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:41.894737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ff20b-63e7-4113-93da-cbfe5c68cd4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=613f21f4b5164e6087e734f9b447e961, user_id=da46b59a0cb5430589aeef1693b2ce50, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:41.896334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 787/1577] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:41.897695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ff20b-63e7-4113-93da-cbfe5c68cd4a admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:41.897903 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27df9af-1913-4c0b-bd83-ad6cafe6b38e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.924299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 791/1578] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:41 2022] PUT /identity/v3/projects/613f21f4b5164e6087e734f9b447e961/users/da46b59a0cb5430589aeef1693b2ce50/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:41.924299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ff20b-63e7-4113-93da-cbfe5c68cd4a admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:41.924783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27df9af-1913-4c0b-bd83-ad6cafe6b38e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:41.928905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:42.005393 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ff20b-63e7-4113-93da-cbfe5c68cd4a admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.011611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec10e1-2746-4c96-b4b1-fd50b33bf5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.012675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec10e1-2746-4c96-b4b1-fd50b33bf5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.012826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7d740-61f3-4331-9f6f-e5ec2e82dd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.013133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec10e1-2746-4c96-b4b1-fd50b33bf5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.013594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec10e1-2746-4c96-b4b1-fd50b33bf5ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.013729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7d740-61f3-4331-9f6f-e5ec2e82dd94 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.014178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7d740-61f3-4331-9f6f-e5ec2e82dd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.015158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7d740-61f3-4331-9f6f-e5ec2e82dd94 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.015410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec10e1-2746-4c96-b4b1-fd50b33bf5ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:42.015920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec10e1-2746-4c96-b4b1-fd50b33bf5ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.016566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7d740-61f3-4331-9f6f-e5ec2e82dd94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.018752 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec10e1-2746-4c96-b4b1-fd50b33bf5ad admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.019912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7d740-61f3-4331-9f6f-e5ec2e82dd94 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.051132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 792/1579] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:42.051132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec10e1-2746-4c96-b4b1-fd50b33bf5ad admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.051595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 788/1580] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:41 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:42.052229 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7d740-61f3-4331-9f6f-e5ec2e82dd94 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.062258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec10e1-2746-4c96-b4b1-fd50b33bf5ad admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.068856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd5773-a6e9-42d8-adb7-c54c8262ff87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.069673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd5773-a6e9-42d8-adb7-c54c8262ff87 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.069982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd5773-a6e9-42d8-adb7-c54c8262ff87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.070261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd5773-a6e9-42d8-adb7-c54c8262ff87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.071030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd5773-a6e9-42d8-adb7-c54c8262ff87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.072734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd5773-a6e9-42d8-adb7-c54c8262ff87 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.086824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 793/1581] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:42.087693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd5773-a6e9-42d8-adb7-c54c8262ff87 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.099617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd5773-a6e9-42d8-adb7-c54c8262ff87 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.104707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3b5c4-cd14-429c-9a95-029905e333e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.105404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3b5c4-cd14-429c-9a95-029905e333e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.105694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3b5c4-cd14-429c-9a95-029905e333e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.105966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3b5c4-cd14-429c-9a95-029905e333e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.106670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3b5c4-cd14-429c-9a95-029905e333e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.109611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3b5c4-cd14-429c-9a95-029905e333e1 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.126934 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7d740-61f3-4331-9f6f-e5ec2e82dd94 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.131668 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab651d-fdb5-4efe-b025-4734f1f7f7d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.132259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab651d-fdb5-4efe-b025-4734f1f7f7d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.132588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab651d-fdb5-4efe-b025-4734f1f7f7d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.132872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab651d-fdb5-4efe-b025-4734f1f7f7d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.144192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 794/1582] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/users => generated 302 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:42.144192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3b5c4-cd14-429c-9a95-029905e333e1 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.144882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab651d-fdb5-4efe-b025-4734f1f7f7d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.145067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab651d-fdb5-4efe-b025-4734f1f7f7d8 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.156443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 789/1583] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:42.157556 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab651d-fdb5-4efe-b025-4734f1f7f7d8 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.162247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3b5c4-cd14-429c-9a95-029905e333e1 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.168080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab651d-fdb5-4efe-b025-4734f1f7f7d8 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.173156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb2cf2-6b19-47bd-ba65-47b5e4c77a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.174707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb2cf2-6b19-47bd-ba65-47b5e4c77a73 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.174921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056a36c-8884-40d9-b806-4249f36ca933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.175421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056a36c-8884-40d9-b806-4249f36ca933 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.175709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056a36c-8884-40d9-b806-4249f36ca933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.175994 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056a36c-8884-40d9-b806-4249f36ca933 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.176579 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb2cf2-6b19-47bd-ba65-47b5e4c77a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.176776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056a36c-8884-40d9-b806-4249f36ca933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.177614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb2cf2-6b19-47bd-ba65-47b5e4c77a73 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.178871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056a36c-8884-40d9-b806-4249f36ca933 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.179090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb2cf2-6b19-47bd-ba65-47b5e4c77a73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.183374 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb2cf2-6b19-47bd-ba65-47b5e4c77a73 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.196224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:42.224014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 795/1584] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/users => generated 287 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:42.226408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9e755-fef3-4947-aaa0-1c48964c931c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.226912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9e755-fef3-4947-aaa0-1c48964c931c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.227516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9e755-fef3-4947-aaa0-1c48964c931c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.231265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 790/1585] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/projects => generated 365 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:42.232938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056a36c-8884-40d9-b806-4249f36ca933 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.273513 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-99c9e755-fef3-4947-aaa0-1c48964c931c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:42.274940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-99c9e755-fef3-4947-aaa0-1c48964c931c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:42.324056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056a36c-8884-40d9-b806-4249f36ca933 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.329044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02c072-55b7-4edb-82d3-779c3955a44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.329741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02c072-55b7-4edb-82d3-779c3955a44d admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.330030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02c072-55b7-4edb-82d3-779c3955a44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.330313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02c072-55b7-4edb-82d3-779c3955a44d admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.331025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02c072-55b7-4edb-82d3-779c3955a44d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.333141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02c072-55b7-4edb-82d3-779c3955a44d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.368262 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 796/1586] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:42.374627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 791/1587] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/projects => generated 391 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:42.374902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9e755-fef3-4947-aaa0-1c48964c931c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.375110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02c072-55b7-4edb-82d3-779c3955a44d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.381314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9e755-fef3-4947-aaa0-1c48964c931c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.456790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:42.477031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45375eee-8cf9-408a-9ba4-11efc8fbbf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.478399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45375eee-8cf9-408a-9ba4-11efc8fbbf0f admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.478803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45375eee-8cf9-408a-9ba4-11efc8fbbf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.479194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45375eee-8cf9-408a-9ba4-11efc8fbbf0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.481060 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45375eee-8cf9-408a-9ba4-11efc8fbbf0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.483117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45375eee-8cf9-408a-9ba4-11efc8fbbf0f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.504002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 797/1588] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:42.506050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45375eee-8cf9-408a-9ba4-11efc8fbbf0f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.507721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02c072-55b7-4edb-82d3-779c3955a44d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.515078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ec297-79ba-4f30-a764-82ea6305acb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.516459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ec297-79ba-4f30-a764-82ea6305acb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.516914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ec297-79ba-4f30-a764-82ea6305acb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.517120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ec297-79ba-4f30-a764-82ea6305acb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.518323 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ec297-79ba-4f30-a764-82ea6305acb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.520065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ec297-79ba-4f30-a764-82ea6305acb6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.533027 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45375eee-8cf9-408a-9ba4-11efc8fbbf0f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.535318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 792/1589] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:42.536305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ec297-79ba-4f30-a764-82ea6305acb6 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.543597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6806670-3b08-4cf0-abd1-89217b0ade33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.544770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6806670-3b08-4cf0-abd1-89217b0ade33 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.545284 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6806670-3b08-4cf0-abd1-89217b0ade33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.546316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6806670-3b08-4cf0-abd1-89217b0ade33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.548763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ec297-79ba-4f30-a764-82ea6305acb6 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.549045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6806670-3b08-4cf0-abd1-89217b0ade33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.552558 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6806670-3b08-4cf0-abd1-89217b0ade33 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.554438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59617c-8005-416e-9b5d-3f86f14b31de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.555135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59617c-8005-416e-9b5d-3f86f14b31de admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.555426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59617c-8005-416e-9b5d-3f86f14b31de admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.555710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59617c-8005-416e-9b5d-3f86f14b31de admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.556508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59617c-8005-416e-9b5d-3f86f14b31de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.558634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59617c-8005-416e-9b5d-3f86f14b31de admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.572198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 793/1590] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:42.573420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59617c-8005-416e-9b5d-3f86f14b31de admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.581568 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 798/1591] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:42.581568 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6806670-3b08-4cf0-abd1-89217b0ade33 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.587854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59617c-8005-416e-9b5d-3f86f14b31de admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.596324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef67eb-eddb-4e14-9fce-f7de88df12c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.597063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef67eb-eddb-4e14-9fce-f7de88df12c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.597366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef67eb-eddb-4e14-9fce-f7de88df12c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.597621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef67eb-eddb-4e14-9fce-f7de88df12c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.598332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef67eb-eddb-4e14-9fce-f7de88df12c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.600170 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6806670-3b08-4cf0-abd1-89217b0ade33 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.600409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef67eb-eddb-4e14-9fce-f7de88df12c0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.611600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909d052-ecd7-4cf0-a6ba-f05d0a67ccfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.614230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909d052-ecd7-4cf0-a6ba-f05d0a67ccfa admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.615218 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909d052-ecd7-4cf0-a6ba-f05d0a67ccfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.616146 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909d052-ecd7-4cf0-a6ba-f05d0a67ccfa admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.617299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909d052-ecd7-4cf0-a6ba-f05d0a67ccfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.621306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909d052-ecd7-4cf0-a6ba-f05d0a67ccfa admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.632267 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 794/1592] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/users => generated 283 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:42.634146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef67eb-eddb-4e14-9fce-f7de88df12c0 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.638644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-84ef67eb-eddb-4e14-9fce-f7de88df12c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:42.707976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 799/1593] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:42 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:42.709696 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909d052-ecd7-4cf0-a6ba-f05d0a67ccfa admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.731095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909d052-ecd7-4cf0-a6ba-f05d0a67ccfa admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.740588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a30ca5-5ebf-4d29-a050-1d0522a50601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.741823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a30ca5-5ebf-4d29-a050-1d0522a50601 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.741982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:42.742331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a30ca5-5ebf-4d29-a050-1d0522a50601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.742994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a30ca5-5ebf-4d29-a050-1d0522a50601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.744246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a30ca5-5ebf-4d29-a050-1d0522a50601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.748391 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a30ca5-5ebf-4d29-a050-1d0522a50601 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.762720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef67eb-eddb-4e14-9fce-f7de88df12c0 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.768838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042c3e0-22b1-4f4c-8086-ff3fedc43f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.770397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042c3e0-22b1-4f4c-8086-ff3fedc43f93 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.770892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042c3e0-22b1-4f4c-8086-ff3fedc43f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.771450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042c3e0-22b1-4f4c-8086-ff3fedc43f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.771778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 800/1594] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:42.773345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a30ca5-5ebf-4d29-a050-1d0522a50601 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.790438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042c3e0-22b1-4f4c-8086-ff3fedc43f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.792655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042c3e0-22b1-4f4c-8086-ff3fedc43f93 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.797326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a30ca5-5ebf-4d29-a050-1d0522a50601 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.813436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f29cb6-6278-47dd-a0a2-b15a307a3f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.815033 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f29cb6-6278-47dd-a0a2-b15a307a3f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.815366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 795/1595] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:42.815519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f29cb6-6278-47dd-a0a2-b15a307a3f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.816290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f29cb6-6278-47dd-a0a2-b15a307a3f34 admin admin] PATH_INFO: `/v3/projects/a4fb92f1e72f4228b7bf298aac07414f/users/cc0017a4dfb440828150d1a89b8635aa/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.817830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042c3e0-22b1-4f4c-8086-ff3fedc43f93 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.836121 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042c3e0-22b1-4f4c-8086-ff3fedc43f93 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.844386 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ddca5-697d-49a4-9c25-75dc0031a4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:42.845632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ddca5-697d-49a4-9c25-75dc0031a4eb admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:42.846343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ddca5-697d-49a4-9c25-75dc0031a4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:42.846792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ddca5-697d-49a4-9c25-75dc0031a4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:42.847911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ddca5-697d-49a4-9c25-75dc0031a4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.849219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f29cb6-6278-47dd-a0a2-b15a307a3f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4fb92f1e72f4228b7bf298aac07414f, user_id=cc0017a4dfb440828150d1a89b8635aa, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:42.850918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ddca5-697d-49a4-9c25-75dc0031a4eb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.854511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f29cb6-6278-47dd-a0a2-b15a307a3f34 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:42.865258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 796/1596] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:42.866671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ddca5-697d-49a4-9c25-75dc0031a4eb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:42.879236 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ddca5-697d-49a4-9c25-75dc0031a4eb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:42.887674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 801/1597] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:42 2022] PUT /identity/v3/projects/a4fb92f1e72f4228b7bf298aac07414f/users/cc0017a4dfb440828150d1a89b8635aa/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:42.889309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f29cb6-6278-47dd-a0a2-b15a307a3f34 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.002243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11244f26-c2ad-45cc-b358-9fbc40364e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.004091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11244f26-c2ad-45cc-b358-9fbc40364e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.004359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11244f26-c2ad-45cc-b358-9fbc40364e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.004845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11244f26-c2ad-45cc-b358-9fbc40364e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.007377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11244f26-c2ad-45cc-b358-9fbc40364e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.010895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11244f26-c2ad-45cc-b358-9fbc40364e72 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.040959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f29cb6-6278-47dd-a0a2-b15a307a3f34 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.041916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 797/1598] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:43.043419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11244f26-c2ad-45cc-b358-9fbc40364e72 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.048548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bc1c9-7dad-457b-ae0f-f2cac3723a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.049605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bc1c9-7dad-457b-ae0f-f2cac3723a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.050132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bc1c9-7dad-457b-ae0f-f2cac3723a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.050901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bc1c9-7dad-457b-ae0f-f2cac3723a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.052608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bc1c9-7dad-457b-ae0f-f2cac3723a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.054397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bc1c9-7dad-457b-ae0f-f2cac3723a4d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.066428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11244f26-c2ad-45cc-b358-9fbc40364e72 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.069696 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 802/1599] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:43.070582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bc1c9-7dad-457b-ae0f-f2cac3723a4d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.082534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bc1c9-7dad-457b-ae0f-f2cac3723a4d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.083097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59f549-ebec-4026-aede-17108bb26e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.084632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59f549-ebec-4026-aede-17108bb26e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.085265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59f549-ebec-4026-aede-17108bb26e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.085893 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59f549-ebec-4026-aede-17108bb26e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.087535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59f549-ebec-4026-aede-17108bb26e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.087762 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ecb6c-55f3-496e-96f9-0e0270f9a659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.088796 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ecb6c-55f3-496e-96f9-0e0270f9a659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.088994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ecb6c-55f3-496e-96f9-0e0270f9a659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.089499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ecb6c-55f3-496e-96f9-0e0270f9a659 admin admin] PATH_INFO: `/v3/projects/61132d14ab244e878995e6b7199dd19f/users/7a798c8bc70c4469972283bf92c39735/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.092983 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59f549-ebec-4026-aede-17108bb26e8a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.115259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 798/1600] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:43.116455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59f549-ebec-4026-aede-17108bb26e8a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.128573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ecb6c-55f3-496e-96f9-0e0270f9a659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61132d14ab244e878995e6b7199dd19f, user_id=7a798c8bc70c4469972283bf92c39735, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.128921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59f549-ebec-4026-aede-17108bb26e8a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.135280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff360d42-0456-4ea7-ab31-579e9b1a88cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.136623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff360d42-0456-4ea7-ab31-579e9b1a88cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.137161 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff360d42-0456-4ea7-ab31-579e9b1a88cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.137389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ecb6c-55f3-496e-96f9-0e0270f9a659 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.137676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff360d42-0456-4ea7-ab31-579e9b1a88cc admin admin] PATH_INFO: `/v3/projects/0e7ddc919f5f4aa2a84749f34d5550e1/users/5621cb413101448c926b37c6519122ac/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.160400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff360d42-0456-4ea7-ab31-579e9b1a88cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e7ddc919f5f4aa2a84749f34d5550e1, user_id=5621cb413101448c926b37c6519122ac, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.161767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-ff360d42-0456-4ea7-ab31-579e9b1a88cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:33:43.161767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: warnings.warn(msg) Jul 21 22:33:43.161767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: Jul 21 22:33:43.164268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 803/1601] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:43 2022] PUT /identity/v3/projects/61132d14ab244e878995e6b7199dd19f/users/7a798c8bc70c4469972283bf92c39735/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:43.164805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff360d42-0456-4ea7-ab31-579e9b1a88cc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.167446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c6028-5075-44d1-948e-271037163ab4 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.167446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c6028-5075-44d1-948e-271037163ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.167737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c6028-5075-44d1-948e-271037163ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.195735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 799/1602] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:43 2022] PUT /identity/v3/projects/0e7ddc919f5f4aa2a84749f34d5550e1/users/5621cb413101448c926b37c6519122ac/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:43.195735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff360d42-0456-4ea7-ab31-579e9b1a88cc admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.222894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-463c6028-5075-44d1-948e-271037163ab4 None None] MFA Rules not processed for user `cc0017a4dfb440828150d1a89b8635aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:43.225222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-463c6028-5075-44d1-948e-271037163ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:43.336839 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:43.344315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff360d42-0456-4ea7-ab31-579e9b1a88cc admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.353236 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16808144-3a5e-4008-8709-e0d8f5500b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.355291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808144-3a5e-4008-8709-e0d8f5500b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.355714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808144-3a5e-4008-8709-e0d8f5500b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.356574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808144-3a5e-4008-8709-e0d8f5500b8e admin admin] PATH_INFO: `/v3/projects/f415c7c6a3de4b0ea481cca7b3cd616c/users/09292da603b944ff92f0022bfb16cd6c/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.381775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 804/1603] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:43.383521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c6028-5075-44d1-948e-271037163ab4 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.399522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c6028-5075-44d1-948e-271037163ab4 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.408329 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63171be2-11a2-4376-be16-771b3dc51735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.409326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63171be2-11a2-4376-be16-771b3dc51735 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.409805 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63171be2-11a2-4376-be16-771b3dc51735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.410306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63171be2-11a2-4376-be16-771b3dc51735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.411776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63171be2-11a2-4376-be16-771b3dc51735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.417667 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16808144-3a5e-4008-8709-e0d8f5500b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f415c7c6a3de4b0ea481cca7b3cd616c, user_id=09292da603b944ff92f0022bfb16cd6c, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.417898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63171be2-11a2-4376-be16-771b3dc51735 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.421555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16808144-3a5e-4008-8709-e0d8f5500b8e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.449251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 805/1604] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:43.457061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63171be2-11a2-4376-be16-771b3dc51735 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.459612 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 800/1605] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:43 2022] PUT /identity/v3/projects/f415c7c6a3de4b0ea481cca7b3cd616c/users/09292da603b944ff92f0022bfb16cd6c/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:43.462113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec42ac-1640-4477-9886-4799af61db5f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.462574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec42ac-1640-4477-9886-4799af61db5f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.463940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec42ac-1640-4477-9886-4799af61db5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.493490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63171be2-11a2-4376-be16-771b3dc51735 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.532015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9eec42ac-1640-4477-9886-4799af61db5f None None] MFA Rules not processed for user `7a798c8bc70c4469972283bf92c39735`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:43.534631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9eec42ac-1640-4477-9886-4799af61db5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:43.547590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:43.657830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad95fa6-fc95-4e4a-831e-f6ef4ed4f9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.659424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad95fa6-fc95-4e4a-831e-f6ef4ed4f9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.659941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad95fa6-fc95-4e4a-831e-f6ef4ed4f9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.660497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad95fa6-fc95-4e4a-831e-f6ef4ed4f9a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.662362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad95fa6-fc95-4e4a-831e-f6ef4ed4f9a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.666398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad95fa6-fc95-4e4a-831e-f6ef4ed4f9a0 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.693187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 806/1606] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:43.702038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470757aa-76ef-4acf-8554-6f48c1c2bdf0 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.702509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470757aa-76ef-4acf-8554-6f48c1c2bdf0 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.703023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470757aa-76ef-4acf-8554-6f48c1c2bdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.712565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 801/1607] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:43.712565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec42ac-1640-4477-9886-4799af61db5f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.741879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec42ac-1640-4477-9886-4799af61db5f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.753205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb0387-0ffc-40fc-ba7b-fc72243491e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.753778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb0387-0ffc-40fc-ba7b-fc72243491e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.754198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb0387-0ffc-40fc-ba7b-fc72243491e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.754960 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb0387-0ffc-40fc-ba7b-fc72243491e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.756850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb0387-0ffc-40fc-ba7b-fc72243491e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:43.761178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb0387-0ffc-40fc-ba7b-fc72243491e9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:43.773418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-470757aa-76ef-4acf-8554-6f48c1c2bdf0 None None] MFA Rules not processed for user `5621cb413101448c926b37c6519122ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:43.775761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-470757aa-76ef-4acf-8554-6f48c1c2bdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:43.820392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 802/1608] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:43 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:43.820392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb0387-0ffc-40fc-ba7b-fc72243491e9 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:43.943785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb0387-0ffc-40fc-ba7b-fc72243491e9 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:43.949995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89fb42-7691-488b-aab2-14439dca7d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:43.951076 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89fb42-7691-488b-aab2-14439dca7d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.951490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89fb42-7691-488b-aab2-14439dca7d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.951892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89fb42-7691-488b-aab2-14439dca7d48 admin admin] PATH_INFO: `/v3/projects/156a0bfceb934d2498348f120b387520/users/b802fcb48ff9485b8c4dc20f9a0a9a48/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.986048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 807/1609] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:43.988657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808c233-7cc7-4ec0-bc93-482fd43b1285 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:43.989179 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808c233-7cc7-4ec0-bc93-482fd43b1285 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:43.989777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808c233-7cc7-4ec0-bc93-482fd43b1285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:43.994663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89fb42-7691-488b-aab2-14439dca7d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=156a0bfceb934d2498348f120b387520, user_id=b802fcb48ff9485b8c4dc20f9a0a9a48, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:44.002804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89fb42-7691-488b-aab2-14439dca7d48 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:44.062414 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 803/1610] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:43 2022] PUT /identity/v3/projects/156a0bfceb934d2498348f120b387520/users/b802fcb48ff9485b8c4dc20f9a0a9a48/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:44.064200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89fb42-7691-488b-aab2-14439dca7d48 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.078737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-8808c233-7cc7-4ec0-bc93-482fd43b1285 None None] MFA Rules not processed for user `09292da603b944ff92f0022bfb16cd6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:44.080369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8808c233-7cc7-4ec0-bc93-482fd43b1285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:44.221317 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:44.240401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89fb42-7691-488b-aab2-14439dca7d48 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:44.251483 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d22c5-fba5-403d-bbc6-04778513253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:44.254154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d22c5-fba5-403d-bbc6-04778513253e admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.254666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d22c5-fba5-403d-bbc6-04778513253e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.255503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d22c5-fba5-403d-bbc6-04778513253e admin admin] PATH_INFO: `/v3/projects/0b2f768290ea43cb98b058067268674d/users/f93bd44f2fbc4be4be905c8b8ae0e343/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.268099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 808/1611] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:44.269805 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808c233-7cc7-4ec0-bc93-482fd43b1285 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.295132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808c233-7cc7-4ec0-bc93-482fd43b1285 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:44.296969 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:44.308775 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d22c5-fba5-403d-bbc6-04778513253e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2f768290ea43cb98b058067268674d, user_id=f93bd44f2fbc4be4be905c8b8ae0e343, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:44.318230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d22c5-fba5-403d-bbc6-04778513253e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:44.321842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ad877-16fe-4b3d-8cbb-56f5285f50f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:44.323140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ad877-16fe-4b3d-8cbb-56f5285f50f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.323551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ad877-16fe-4b3d-8cbb-56f5285f50f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.324055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ad877-16fe-4b3d-8cbb-56f5285f50f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.325457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ad877-16fe-4b3d-8cbb-56f5285f50f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:44.329735 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ad877-16fe-4b3d-8cbb-56f5285f50f8 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:44.365595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 804/1612] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:44 2022] PUT /identity/v3/projects/0b2f768290ea43cb98b058067268674d/users/f93bd44f2fbc4be4be905c8b8ae0e343/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:44.367344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d22c5-fba5-403d-bbc6-04778513253e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.393966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 809/1613] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:44 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:44.397363 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77132363-4d39-4080-90fb-0c56a5f22858 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.397965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77132363-4d39-4080-90fb-0c56a5f22858 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.398614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77132363-4d39-4080-90fb-0c56a5f22858 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.402392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 810/1614] 10.222.0.42 () {54 vars in 962 bytes} [Thu Jul 21 22:33:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:33:44.405680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cd63f-4208-41d9-a294-19cc3d036781 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.407505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cd63f-4208-41d9-a294-19cc3d036781 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.408184 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cd63f-4208-41d9-a294-19cc3d036781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.468432 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-d87cd63f-4208-41d9-a294-19cc3d036781 None None] MFA Rules not processed for user `b802fcb48ff9485b8c4dc20f9a0a9a48`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:44.471616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d87cd63f-4208-41d9-a294-19cc3d036781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:44.532397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d22c5-fba5-403d-bbc6-04778513253e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:44.652703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 811/1615] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:44.658212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cd63f-4208-41d9-a294-19cc3d036781 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.696370 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7d32f-eeda-499a-b319-927af861a58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:44.697870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7d32f-eeda-499a-b319-927af861a58f admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.698793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7d32f-eeda-499a-b319-927af861a58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.699364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7d32f-eeda-499a-b319-927af861a58f admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.700795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7d32f-eeda-499a-b319-927af861a58f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:44.704741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7d32f-eeda-499a-b319-927af861a58f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:44.776352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 805/1616] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:44 2022] POST /identity/v3/users => generated 295 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:44.779182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f140cc-a6bf-4821-8a5f-989e244c332a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.779684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f140cc-a6bf-4821-8a5f-989e244c332a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.781401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f140cc-a6bf-4821-8a5f-989e244c332a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.820435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b6f140cc-a6bf-4821-8a5f-989e244c332a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:44.828458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b6f140cc-a6bf-4821-8a5f-989e244c332a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:44.830714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cd63f-4208-41d9-a294-19cc3d036781 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:44.841330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 806/1617] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:44.842142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f140cc-a6bf-4821-8a5f-989e244c332a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.860456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f140cc-a6bf-4821-8a5f-989e244c332a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:44.863635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743a8c1-d147-4102-8c0a-488864e96506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:44.865038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743a8c1-d147-4102-8c0a-488864e96506 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.865686 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743a8c1-d147-4102-8c0a-488864e96506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.866307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743a8c1-d147-4102-8c0a-488864e96506 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.866819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bb4c5-0765-4018-a1b7-8ca7bb7f5a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:44.868380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bb4c5-0765-4018-a1b7-8ca7bb7f5a38 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.868380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bb4c5-0765-4018-a1b7-8ca7bb7f5a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.868778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743a8c1-d147-4102-8c0a-488864e96506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:44.868943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bb4c5-0765-4018-a1b7-8ca7bb7f5a38 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.869257 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bb4c5-0765-4018-a1b7-8ca7bb7f5a38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:44.872183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743a8c1-d147-4102-8c0a-488864e96506 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:44.872430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bb4c5-0765-4018-a1b7-8ca7bb7f5a38 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:44.905458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 807/1618] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:44 2022] POST /identity/v3/users => generated 296 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:44.907703 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f10c21-2853-4047-894b-621bf4e5860c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:44.908130 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f10c21-2853-4047-894b-621bf4e5860c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:44.908551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f10c21-2853-4047-894b-621bf4e5860c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:44.970526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 812/1619] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:44 2022] POST /identity/v3/projects => generated 405 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:44.972977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743a8c1-d147-4102-8c0a-488864e96506 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:44.973400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-83f10c21-2853-4047-894b-621bf4e5860c None None] MFA Rules not processed for user `c77f6ccc7cb84c04ac532007a04a0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:44.974729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-83f10c21-2853-4047-894b-621bf4e5860c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:45.036654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:45.093227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 808/1620] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Jul 21 22:33:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:45.094870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f10c21-2853-4047-894b-621bf4e5860c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.141353 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:45.176214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743a8c1-d147-4102-8c0a-488864e96506 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.182635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f23097-7db0-4dd9-8f20-21cd6ae9cf11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.183327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f23097-7db0-4dd9-8f20-21cd6ae9cf11 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.183624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f23097-7db0-4dd9-8f20-21cd6ae9cf11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.183901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f23097-7db0-4dd9-8f20-21cd6ae9cf11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.258248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f10c21-2853-4047-894b-621bf4e5860c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.264821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecebd95-c6ec-433c-a95a-9e5fe71994ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.265920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecebd95-c6ec-433c-a95a-9e5fe71994ff admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.266211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecebd95-c6ec-433c-a95a-9e5fe71994ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.266491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecebd95-c6ec-433c-a95a-9e5fe71994ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.267757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecebd95-c6ec-433c-a95a-9e5fe71994ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.269699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecebd95-c6ec-433c-a95a-9e5fe71994ff admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.289152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 809/1621] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:45.290548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecebd95-c6ec-433c-a95a-9e5fe71994ff admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.308855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecebd95-c6ec-433c-a95a-9e5fe71994ff admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.318693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39170d2b-2ce7-45a8-8413-fa9626ad61c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.320205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39170d2b-2ce7-45a8-8413-fa9626ad61c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.320728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39170d2b-2ce7-45a8-8413-fa9626ad61c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.321165 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39170d2b-2ce7-45a8-8413-fa9626ad61c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.322462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39170d2b-2ce7-45a8-8413-fa9626ad61c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.325569 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39170d2b-2ce7-45a8-8413-fa9626ad61c2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.345524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f23097-7db0-4dd9-8f20-21cd6ae9cf11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.350651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f23097-7db0-4dd9-8f20-21cd6ae9cf11 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.353709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 810/1622] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:45.355499 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39170d2b-2ce7-45a8-8413-fa9626ad61c2 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.375008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 813/1623] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:45.376226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f23097-7db0-4dd9-8f20-21cd6ae9cf11 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.394010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f23097-7db0-4dd9-8f20-21cd6ae9cf11 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.400947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb1b97-8f50-405e-9dc3-e3fa7c4a2a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.401710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb1b97-8f50-405e-9dc3-e3fa7c4a2a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.402005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb1b97-8f50-405e-9dc3-e3fa7c4a2a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.402288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb1b97-8f50-405e-9dc3-e3fa7c4a2a31 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.403082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb1b97-8f50-405e-9dc3-e3fa7c4a2a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.406188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb1b97-8f50-405e-9dc3-e3fa7c4a2a31 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.442586 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 814/1624] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:45 2022] POST /identity/v3/users => generated 309 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:45.443412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb1b97-8f50-405e-9dc3-e3fa7c4a2a31 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.482241 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb1b97-8f50-405e-9dc3-e3fa7c4a2a31 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.501384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82fe2b-93ca-4cdf-aedf-8d89bed964ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.502657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82fe2b-93ca-4cdf-aedf-8d89bed964ac service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.502948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82fe2b-93ca-4cdf-aedf-8d89bed964ac service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.503671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82fe2b-93ca-4cdf-aedf-8d89bed964ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.534189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39170d2b-2ce7-45a8-8413-fa9626ad61c2 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.545095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fd5d4-dfe0-4765-9097-6a57455048c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.546029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fd5d4-dfe0-4765-9097-6a57455048c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.546360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fd5d4-dfe0-4765-9097-6a57455048c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.546679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fd5d4-dfe0-4765-9097-6a57455048c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.568619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0fd5d4-dfe0-4765-9097-6a57455048c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.571625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0fd5d4-dfe0-4765-9097-6a57455048c3 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.597015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 811/1625] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:45.597959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fd5d4-dfe0-4765-9097-6a57455048c3 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.622746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fd5d4-dfe0-4765-9097-6a57455048c3 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.647456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ce326-68ee-45ad-8177-c61b5655f088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.649640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ce326-68ee-45ad-8177-c61b5655f088 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.652470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ce326-68ee-45ad-8177-c61b5655f088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.652784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ce326-68ee-45ad-8177-c61b5655f088 admin admin] PATH_INFO: `/v3/projects/ae0f5817de184b6797186f21458b5176/users/501569bb389f42e1893c9e993aa8f837/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.656609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82fe2b-93ca-4cdf-aedf-8d89bed964ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.658602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82fe2b-93ca-4cdf-aedf-8d89bed964ac service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.687701 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 815/1626] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:45.689315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82fe2b-93ca-4cdf-aedf-8d89bed964ac service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.693407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ce326-68ee-45ad-8177-c61b5655f088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae0f5817de184b6797186f21458b5176, user_id=501569bb389f42e1893c9e993aa8f837, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.696178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ce326-68ee-45ad-8177-c61b5655f088 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.716958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82fe2b-93ca-4cdf-aedf-8d89bed964ac service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.736754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 812/1627] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:45 2022] PUT /identity/v3/projects/ae0f5817de184b6797186f21458b5176/users/501569bb389f42e1893c9e993aa8f837/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:45.737735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ce326-68ee-45ad-8177-c61b5655f088 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.878088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:45.891047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb636a-5dfc-4a3c-90bf-2e6e20a71bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.892057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb636a-5dfc-4a3c-90bf-2e6e20a71bea admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.892279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb636a-5dfc-4a3c-90bf-2e6e20a71bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.892586 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb636a-5dfc-4a3c-90bf-2e6e20a71bea admin admin] PATH_INFO: `/v3/projects/9e5c871bb5664dba96c9f7692e5285a5/users/3b755eb3682b4686a68a5fcb4bb67266/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.919424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ce326-68ee-45ad-8177-c61b5655f088 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:45.931634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f462ce-4134-4c0c-b59b-71f9d2c03ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:45.932338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f462ce-4134-4c0c-b59b-71f9d2c03ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:45.932663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f462ce-4134-4c0c-b59b-71f9d2c03ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:45.932942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f462ce-4134-4c0c-b59b-71f9d2c03ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:45.935495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f462ce-4134-4c0c-b59b-71f9d2c03ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.938567 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb636a-5dfc-4a3c-90bf-2e6e20a71bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5c871bb5664dba96c9f7692e5285a5, user_id=3b755eb3682b4686a68a5fcb4bb67266, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:45.942183 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f462ce-4134-4c0c-b59b-71f9d2c03ca7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.950422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb636a-5dfc-4a3c-90bf-2e6e20a71bea admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:45.979479 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 813/1628] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:45.981314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f462ce-4134-4c0c-b59b-71f9d2c03ca7 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:45.989140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 816/1629] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:45 2022] PUT /identity/v3/projects/9e5c871bb5664dba96c9f7692e5285a5/users/3b755eb3682b4686a68a5fcb4bb67266/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:45.990057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb636a-5dfc-4a3c-90bf-2e6e20a71bea admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:46.007724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:46.021724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f462ce-4134-4c0c-b59b-71f9d2c03ca7 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:46.216630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb636a-5dfc-4a3c-90bf-2e6e20a71bea admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:46.216630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed520e21-3834-45c3-a89a-1afca1ea4ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:46.220465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c0054-ea75-4d57-ada4-070a08635942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:46.220465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c0054-ea75-4d57-ada4-070a08635942 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:46.220465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c0054-ea75-4d57-ada4-070a08635942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:46.220465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c0054-ea75-4d57-ada4-070a08635942 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:46.222800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c0054-ea75-4d57-ada4-070a08635942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:46.223057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed520e21-3834-45c3-a89a-1afca1ea4ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:46.224192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed520e21-3834-45c3-a89a-1afca1ea4ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:46.225328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed520e21-3834-45c3-a89a-1afca1ea4ca8 admin admin] PATH_INFO: `/v3/projects/f3e0b7d190374b9c92fc7990c103f330/users/9e2155cff0ba40598d4c009d91b92194/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:46.227878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c0054-ea75-4d57-ada4-070a08635942 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:46.269755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 814/1630] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:46.270808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c0054-ea75-4d57-ada4-070a08635942 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:46.282203 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed520e21-3834-45c3-a89a-1afca1ea4ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3e0b7d190374b9c92fc7990c103f330, user_id=9e2155cff0ba40598d4c009d91b92194, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:46.285244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed520e21-3834-45c3-a89a-1afca1ea4ca8 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:46.300106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c0054-ea75-4d57-ada4-070a08635942 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:46.318176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911d26e-8d2d-4b02-a7b7-c732b1ab44ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:46.319433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911d26e-8d2d-4b02-a7b7-c732b1ab44ac admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:46.322564 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 817/1631] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:45 2022] PUT /identity/v3/projects/f3e0b7d190374b9c92fc7990c103f330/users/9e2155cff0ba40598d4c009d91b92194/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:46.322758 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911d26e-8d2d-4b02-a7b7-c732b1ab44ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:46.322758 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911d26e-8d2d-4b02-a7b7-c732b1ab44ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:46.322973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed520e21-3834-45c3-a89a-1afca1ea4ca8 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:46.323219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911d26e-8d2d-4b02-a7b7-c732b1ab44ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:46.326516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911d26e-8d2d-4b02-a7b7-c732b1ab44ac admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:46.365838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 815/1632] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:46.367754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911d26e-8d2d-4b02-a7b7-c732b1ab44ac admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:46.415798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911d26e-8d2d-4b02-a7b7-c732b1ab44ac admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:46.542924 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed520e21-3834-45c3-a89a-1afca1ea4ca8 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:46.565095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3632645-787d-45e7-b5ff-a1fe1b042f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:46.566606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3632645-787d-45e7-b5ff-a1fe1b042f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:46.566606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3632645-787d-45e7-b5ff-a1fe1b042f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:46.566606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3632645-787d-45e7-b5ff-a1fe1b042f4b admin admin] PATH_INFO: `/v3/projects/ae0f5817de184b6797186f21458b5176/users/501569bb389f42e1893c9e993aa8f837/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:46.570351 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515d3cd-4d3c-4b07-bf32-5cc383d0b9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:46.571148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515d3cd-4d3c-4b07-bf32-5cc383d0b9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:46.571444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515d3cd-4d3c-4b07-bf32-5cc383d0b9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:46.571781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515d3cd-4d3c-4b07-bf32-5cc383d0b9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:46.572650 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515d3cd-4d3c-4b07-bf32-5cc383d0b9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:46.574542 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3632645-787d-45e7-b5ff-a1fe1b042f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae0f5817de184b6797186f21458b5176, user_id=501569bb389f42e1893c9e993aa8f837, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:46.574728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515d3cd-4d3c-4b07-bf32-5cc383d0b9cc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:46.577387 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3632645-787d-45e7-b5ff-a1fe1b042f4b admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:46.594451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 816/1633] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:46.596256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515d3cd-4d3c-4b07-bf32-5cc383d0b9cc admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:46.615967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515d3cd-4d3c-4b07-bf32-5cc383d0b9cc admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:46.618142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 818/1634] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:46 2022] PUT /identity/v3/projects/ae0f5817de184b6797186f21458b5176/users/501569bb389f42e1893c9e993aa8f837/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:46.619226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3632645-787d-45e7-b5ff-a1fe1b042f4b admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:46.878139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51631874-5ff9-438e-b9f0-4ab3b0d78495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:46.881373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51631874-5ff9-438e-b9f0-4ab3b0d78495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:46.882008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51631874-5ff9-438e-b9f0-4ab3b0d78495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:46.882466 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51631874-5ff9-438e-b9f0-4ab3b0d78495 admin admin] PATH_INFO: `/v3/projects/9e5c871bb5664dba96c9f7692e5285a5/users/3b755eb3682b4686a68a5fcb4bb67266/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:46.886942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3632645-787d-45e7-b5ff-a1fe1b042f4b admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:46.887214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51631874-5ff9-438e-b9f0-4ab3b0d78495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5c871bb5664dba96c9f7692e5285a5, user_id=3b755eb3682b4686a68a5fcb4bb67266, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:46.895772 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51631874-5ff9-438e-b9f0-4ab3b0d78495 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:46.899182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff042b-ee1d-449f-b3c5-0397c88d620e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:46.904333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:46.910347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff042b-ee1d-449f-b3c5-0397c88d620e admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:46.910758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff042b-ee1d-449f-b3c5-0397c88d620e admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:46.911202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff042b-ee1d-449f-b3c5-0397c88d620e admin admin] PATH_INFO: `/v3/projects/f3e0b7d190374b9c92fc7990c103f330/users/9e2155cff0ba40598d4c009d91b92194/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:46.919616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff042b-ee1d-449f-b3c5-0397c88d620e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3e0b7d190374b9c92fc7990c103f330, user_id=9e2155cff0ba40598d4c009d91b92194, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:46.927743 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 817/1635] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:46 2022] PUT /identity/v3/projects/9e5c871bb5664dba96c9f7692e5285a5/users/3b755eb3682b4686a68a5fcb4bb67266/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:46.930367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed7b91-5144-47aa-9e19-45165e42d736 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:46.930667 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed7b91-5144-47aa-9e19-45165e42d736 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:46.930940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed7b91-5144-47aa-9e19-45165e42d736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:46.949987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff042b-ee1d-449f-b3c5-0397c88d620e admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:47.017607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 819/1636] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:46 2022] PUT /identity/v3/projects/f3e0b7d190374b9c92fc7990c103f330/users/9e2155cff0ba40598d4c009d91b92194/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:47.019287 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-eeed7b91-5144-47aa-9e19-45165e42d736 None None] MFA Rules not processed for user `501569bb389f42e1893c9e993aa8f837`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:47.021070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-eeed7b91-5144-47aa-9e19-45165e42d736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:47.022457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c556-f3cb-4a92-ac01-c8771ae677d5 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:47.033302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c556-f3cb-4a92-ac01-c8771ae677d5 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:47.034010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c556-f3cb-4a92-ac01-c8771ae677d5 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:47.036817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 820/1637] 10.222.0.42 () {54 vars in 912 bytes} [Thu Jul 21 22:33:47 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:33:47.046956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befcc56-062b-4fab-8062-b4a5324cd424 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:47.047656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befcc56-062b-4fab-8062-b4a5324cd424 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:47.048264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befcc56-062b-4fab-8062-b4a5324cd424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:47.167077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:47.190031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-2befcc56-062b-4fab-8062-b4a5324cd424 None None] MFA Rules not processed for user `3b755eb3682b4686a68a5fcb4bb67266`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:47.193330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2befcc56-062b-4fab-8062-b4a5324cd424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:47.207548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 818/1638] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:47.207548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d88e05-49b6-444f-b665-9cc56af24096 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:47.207548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d88e05-49b6-444f-b665-9cc56af24096 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:47.207548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d88e05-49b6-444f-b665-9cc56af24096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:47.326173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-54d88e05-49b6-444f-b665-9cc56af24096 None None] MFA Rules not processed for user `9e2155cff0ba40598d4c009d91b92194`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:47.329141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-54d88e05-49b6-444f-b665-9cc56af24096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:47.541573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 821/1639] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:47.562470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a31a0-2195-433f-8f32-2c09dcf844f7 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:47.563182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a31a0-2195-433f-8f32-2c09dcf844f7 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:47.563688 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a31a0-2195-433f-8f32-2c09dcf844f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:47.661151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7e1a31a0-2195-433f-8f32-2c09dcf844f7 None None] MFA Rules not processed for user `c77f6ccc7cb84c04ac532007a04a0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:47.661407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 819/1640] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:47 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:47.663452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7e1a31a0-2195-433f-8f32-2c09dcf844f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:47.671354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d88e05-49b6-444f-b665-9cc56af24096 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:47.851774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d88e05-49b6-444f-b665-9cc56af24096 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:47.860804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d78de-4229-4225-9235-9d7b4869d9ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:47.861491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d78de-4229-4225-9235-9d7b4869d9ba service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:47.861923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d78de-4229-4225-9235-9d7b4869d9ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:47.862086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d78de-4229-4225-9235-9d7b4869d9ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:47.899100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 822/1641] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 22:33:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:47.903042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a31a0-2195-433f-8f32-2c09dcf844f7 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:47.919205 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a31a0-2195-433f-8f32-2c09dcf844f7 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:47.925446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:47.934423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33285d53-3cf5-49c5-9eeb-475eacfe26a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:47.936964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33285d53-3cf5-49c5-9eeb-475eacfe26a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:47.937841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33285d53-3cf5-49c5-9eeb-475eacfe26a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:47.938695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33285d53-3cf5-49c5-9eeb-475eacfe26a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:47.963937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33285d53-3cf5-49c5-9eeb-475eacfe26a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:47.967369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33285d53-3cf5-49c5-9eeb-475eacfe26a7 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:47.999953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 823/1642] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:48.017993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d78de-4229-4225-9235-9d7b4869d9ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:48.020915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d78de-4229-4225-9235-9d7b4869d9ba service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:48.054996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 820/1643] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:48.087983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33285d53-3cf5-49c5-9eeb-475eacfe26a7 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:48.102217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33285d53-3cf5-49c5-9eeb-475eacfe26a7 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:48.111066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4f9a9-0f85-45cc-9ec7-320b85fa144f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:48.113836 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4f9a9-0f85-45cc-9ec7-320b85fa144f service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:48.114294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4f9a9-0f85-45cc-9ec7-320b85fa144f service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:48.114695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4f9a9-0f85-45cc-9ec7-320b85fa144f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:48.131341 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4f9a9-0f85-45cc-9ec7-320b85fa144f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:48.138300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4f9a9-0f85-45cc-9ec7-320b85fa144f service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:48.153501 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 824/1644] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:48 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:48.515747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d78de-4229-4225-9235-9d7b4869d9ba service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:48.557002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d78de-4229-4225-9235-9d7b4869d9ba service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:48.587585 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4605ff-f143-4979-9321-8119e4a15a29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:48.591979 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4605ff-f143-4979-9321-8119e4a15a29 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:48.593539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4605ff-f143-4979-9321-8119e4a15a29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:48.595427 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4605ff-f143-4979-9321-8119e4a15a29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:48.662347 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4605ff-f143-4979-9321-8119e4a15a29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:48.669162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4605ff-f143-4979-9321-8119e4a15a29 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:48.773923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 821/1645] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:49.438668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4f9a9-0f85-45cc-9ec7-320b85fa144f service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:49.448693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4f9a9-0f85-45cc-9ec7-320b85fa144f service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:49.461027 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a4b12-f9e8-44c9-b552-baca0c803c3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:49.462102 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a4b12-f9e8-44c9-b552-baca0c803c3b service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.462570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a4b12-f9e8-44c9-b552-baca0c803c3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.465594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a4b12-f9e8-44c9-b552-baca0c803c3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:49.562731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564f18f-2eaa-48cc-b506-b6c1f8aacca4 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.564236 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564f18f-2eaa-48cc-b506-b6c1f8aacca4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.565766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564f18f-2eaa-48cc-b506-b6c1f8aacca4 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:49.569442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 822/1646] 10.222.0.42 () {54 vars in 962 bytes} [Thu Jul 21 22:33:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:33:49.587500 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730d13b-bb95-4591-8ed2-dd122fad1673 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.588359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730d13b-bb95-4591-8ed2-dd122fad1673 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.589311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730d13b-bb95-4591-8ed2-dd122fad1673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:49.597885 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a4b12-f9e8-44c9-b552-baca0c803c3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:49.600158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a4b12-f9e8-44c9-b552-baca0c803c3b service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:49.613070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 825/1647] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:49.673283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e730d13b-bb95-4591-8ed2-dd122fad1673 None None] MFA Rules not processed for user `c77f6ccc7cb84c04ac532007a04a0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:49.676471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e730d13b-bb95-4591-8ed2-dd122fad1673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:49.703832 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 823/1648] 10.222.0.42 () {58 vars in 1059 bytes} [Thu Jul 21 22:33:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:49.705075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48313ae-6715-4c26-b43f-ec702254ad77 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.705561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48313ae-6715-4c26-b43f-ec702254ad77 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.706000 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48313ae-6715-4c26-b43f-ec702254ad77 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:49.708545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 826/1649] 10.222.0.42 () {54 vars in 912 bytes} [Thu Jul 21 22:33:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:33:49.720793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5a9d0-10df-477d-96b6-f506145ed270 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.721798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5a9d0-10df-477d-96b6-f506145ed270 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.724746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5a9d0-10df-477d-96b6-f506145ed270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:49.782787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-29d5a9d0-10df-477d-96b6-f506145ed270 None None] MFA Rules not processed for user `c77f6ccc7cb84c04ac532007a04a0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:49.784337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-29d5a9d0-10df-477d-96b6-f506145ed270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:49.816543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 824/1650] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 22:33:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:49.914278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48313ae-6715-4c26-b43f-ec702254ad77 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:49.922346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:49.932404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48313ae-6715-4c26-b43f-ec702254ad77 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:49.943145 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048eff3-5c18-4d68-876a-ba81d2dc1b4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:49.944036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048eff3-5c18-4d68-876a-ba81d2dc1b4c service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:49.944328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048eff3-5c18-4d68-876a-ba81d2dc1b4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:49.944651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048eff3-5c18-4d68-876a-ba81d2dc1b4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:50.005434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048eff3-5c18-4d68-876a-ba81d2dc1b4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:50.009093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048eff3-5c18-4d68-876a-ba81d2dc1b4c service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:50.074981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 827/1651] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:33:50.237234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5a9d0-10df-477d-96b6-f506145ed270 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:50.544747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5a9d0-10df-477d-96b6-f506145ed270 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:50.569819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:50.571158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:50.571824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:50.574886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] PATH_INFO: `/v3/users/09292da603b944ff92f0022bfb16cd6c` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:50.583192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] RBAC: Authorizing `identity:delete_user(user_id=09292da603b944ff92f0022bfb16cd6c)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:50.585124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:33:50.585124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: warnings.warn(msg) Jul 21 22:33:50.585124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: Jul 21 22:33:50.588135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:50.771726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09292da603b944ff92f0022bfb16cd6c'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:50.804818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09292da603b944ff92f0022bfb16cd6c'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:50.862609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09292da603b944ff92f0022bfb16cd6c'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:50.883840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09292da603b944ff92f0022bfb16cd6c'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:50.884990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09292da603b944ff92f0022bfb16cd6c'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:50.900719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 825/1652] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:50 2022] DELETE /identity/v3/users/09292da603b944ff92f0022bfb16cd6c => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:50.902227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aed5b8-2950-408e-ab35-297040699c0c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:50.902704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aed5b8-2950-408e-ab35-297040699c0c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:50.902773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aed5b8-2950-408e-ab35-297040699c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:50.951860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-44aed5b8-2950-408e-ab35-297040699c0c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:50.953172 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-44aed5b8-2950-408e-ab35-297040699c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:51.127396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 828/1653] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:51.146850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:51.150130 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:51.197012 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:51.323544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c0775-9cd2-4573-a73b-f0889574a99e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:51.339516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd287d82-ba2b-4feb-af0d-b008693557f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:51.340522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd287d82-ba2b-4feb-af0d-b008693557f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:51.341245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd287d82-ba2b-4feb-af0d-b008693557f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:51.341798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd287d82-ba2b-4feb-af0d-b008693557f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:51.423631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd287d82-ba2b-4feb-af0d-b008693557f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:51.425399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd287d82-ba2b-4feb-af0d-b008693557f9 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:51.463249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 826/1654] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:53.466251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aed5b8-2950-408e-ab35-297040699c0c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:53.471701 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-44aed5b8-2950-408e-ab35-297040699c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:53.516313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aed5b8-2950-408e-ab35-297040699c0c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:53.550665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:53.551753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:53.552055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:53.552334 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] PATH_INFO: `/v3/projects/f415c7c6a3de4b0ea481cca7b3cd616c` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:53.564194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f415c7c6a3de4b0ea481cca7b3cd616c)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:53.568919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: WARNING py.warnings [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:33:53.568919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: warnings.warn(msg) Jul 21 22:33:53.568919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: Jul 21 22:33:53.570294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:53.861670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f415c7c6a3de4b0ea481cca7b3cd616c', 'request_id': 'req-8fa3001c-4acf-478b-a861-de255d719770'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:53.886045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f415c7c6a3de4b0ea481cca7b3cd616c', 'request_id': 'req-8fa3001c-4acf-478b-a861-de255d719770'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:53.899681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f415c7c6a3de4b0ea481cca7b3cd616c', 'request_id': 'req-8fa3001c-4acf-478b-a861-de255d719770'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:53.936965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] The token cache is being invalidate because project f415c7c6a3de4b0ea481cca7b3cd616c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:53.937290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:53.938248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:53.943371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 829/1655] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:53 2022] DELETE /identity/v3/projects/f415c7c6a3de4b0ea481cca7b3cd616c => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:53.944299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd287d82-ba2b-4feb-af0d-b008693557f9 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:53.958952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd287d82-ba2b-4feb-af0d-b008693557f9 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:54.097237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f95766-3e18-4eb3-a39d-109182f0994b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:54.098202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f95766-3e18-4eb3-a39d-109182f0994b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:54.098490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f95766-3e18-4eb3-a39d-109182f0994b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:54.098774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f95766-3e18-4eb3-a39d-109182f0994b admin admin] PATH_INFO: `/v3/users/9e2155cff0ba40598d4c009d91b92194` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:54.102501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f95766-3e18-4eb3-a39d-109182f0994b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e2155cff0ba40598d4c009d91b92194)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:54.104559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f95766-3e18-4eb3-a39d-109182f0994b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:54.212557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f0f95766-3e18-4eb3-a39d-109182f0994b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e2155cff0ba40598d4c009d91b92194'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:54.231570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f0f95766-3e18-4eb3-a39d-109182f0994b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e2155cff0ba40598d4c009d91b92194'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:54.272106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f0f95766-3e18-4eb3-a39d-109182f0994b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e2155cff0ba40598d4c009d91b92194'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:54.290173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f0f95766-3e18-4eb3-a39d-109182f0994b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e2155cff0ba40598d4c009d91b92194'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:54.291519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f0f95766-3e18-4eb3-a39d-109182f0994b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e2155cff0ba40598d4c009d91b92194'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:54.302011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 827/1656] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:53 2022] DELETE /identity/v3/users/9e2155cff0ba40598d4c009d91b92194 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:33:55.190238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:55.193737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:55.240299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5871e-6e26-4b69-9383-965f5dbc278d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:55.241155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5871e-6e26-4b69-9383-965f5dbc278d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:55.241551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5871e-6e26-4b69-9383-965f5dbc278d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:55.281206 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3001c-4acf-478b-a861-de255d719770 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:55.344941 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-98c5871e-6e26-4b69-9383-965f5dbc278d None None] MFA Rules not processed for user `da46b59a0cb5430589aeef1693b2ce50`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:55.347092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-98c5871e-6e26-4b69-9383-965f5dbc278d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:55.453206 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:55.472960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7332a-ac10-4e34-8e4f-693ccdf831b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:55.474377 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7332a-ac10-4e34-8e4f-693ccdf831b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:55.474850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7332a-ac10-4e34-8e4f-693ccdf831b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:55.475313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7332a-ac10-4e34-8e4f-693ccdf831b3 admin admin] PATH_INFO: `/v3/projects/f3e0b7d190374b9c92fc7990c103f330` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:55.489708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7332a-ac10-4e34-8e4f-693ccdf831b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3e0b7d190374b9c92fc7990c103f330)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:55.494106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7332a-ac10-4e34-8e4f-693ccdf831b3 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:55.575646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:55.616321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 828/1657] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:55.652125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5871e-6e26-4b69-9383-965f5dbc278d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:55.653410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-18f7332a-ac10-4e34-8e4f-693ccdf831b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3e0b7d190374b9c92fc7990c103f330', 'request_id': 'req-18f7332a-ac10-4e34-8e4f-693ccdf831b3'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.656126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-98c5871e-6e26-4b69-9383-965f5dbc278d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:55.672061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-18f7332a-ac10-4e34-8e4f-693ccdf831b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3e0b7d190374b9c92fc7990c103f330', 'request_id': 'req-18f7332a-ac10-4e34-8e4f-693ccdf831b3'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.683917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-18f7332a-ac10-4e34-8e4f-693ccdf831b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3e0b7d190374b9c92fc7990c103f330', 'request_id': 'req-18f7332a-ac10-4e34-8e4f-693ccdf831b3'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.705698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-18f7332a-ac10-4e34-8e4f-693ccdf831b3 admin admin] The token cache is being invalidate because project f3e0b7d190374b9c92fc7990c103f330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:33:55.706276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-18f7332a-ac10-4e34-8e4f-693ccdf831b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.707628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-18f7332a-ac10-4e34-8e4f-693ccdf831b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:55.710437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 830/1658] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:33:55 2022] DELETE /identity/v3/projects/f3e0b7d190374b9c92fc7990c103f330 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:55.715524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77291d8b-b1b5-48f0-9e09-1557a4712d43 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:55.717729 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77291d8b-b1b5-48f0-9e09-1557a4712d43 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:55.718250 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77291d8b-b1b5-48f0-9e09-1557a4712d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:55.763618 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-77291d8b-b1b5-48f0-9e09-1557a4712d43 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:55.766716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-77291d8b-b1b5-48f0-9e09-1557a4712d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:55.878661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5871e-6e26-4b69-9383-965f5dbc278d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:55.921462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 831/1659] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:55.940561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95819b4a-2b17-46ce-94b3-5988de398a05 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:55.941484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95819b4a-2b17-46ce-94b3-5988de398a05 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:55.942186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95819b4a-2b17-46ce-94b3-5988de398a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:55.993214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd758b6-6725-41c9-8f21-a48e6bdc7075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:55.993891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd758b6-6725-41c9-8f21-a48e6bdc7075 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:55.996773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd758b6-6725-41c9-8f21-a48e6bdc7075 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:55.997121 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd758b6-6725-41c9-8f21-a48e6bdc7075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:56.000435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6cd758b6-6725-41c9-8f21-a48e6bdc7075 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:56.008935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-95819b4a-2b17-46ce-94b3-5988de398a05 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:56.013228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-95819b4a-2b17-46ce-94b3-5988de398a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:56.041107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 832/1660] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:56.043535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95819b4a-2b17-46ce-94b3-5988de398a05 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:56.067901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95819b4a-2b17-46ce-94b3-5988de398a05 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:56.080594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2d59f-a439-4a23-8713-77bb232c86e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:56.081309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2d59f-a439-4a23-8713-77bb232c86e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:56.081597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2d59f-a439-4a23-8713-77bb232c86e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:56.081881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2d59f-a439-4a23-8713-77bb232c86e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:56.082659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2d59f-a439-4a23-8713-77bb232c86e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:33:56.082999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2d59f-a439-4a23-8713-77bb232c86e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:56.084670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b2d59f-a439-4a23-8713-77bb232c86e4 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:56.121142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 833/1661] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:33:56.125593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2d59f-a439-4a23-8713-77bb232c86e4 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:56.147208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2d59f-a439-4a23-8713-77bb232c86e4 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:56.173116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd758b6-6725-41c9-8f21-a48e6bdc7075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:56.174909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd758b6-6725-41c9-8f21-a48e6bdc7075 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:56.181338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73451ed7-7cda-40b6-8acb-4bc370c35388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:56.182062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73451ed7-7cda-40b6-8acb-4bc370c35388 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:56.182333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73451ed7-7cda-40b6-8acb-4bc370c35388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:56.182627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73451ed7-7cda-40b6-8acb-4bc370c35388 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:56.183403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73451ed7-7cda-40b6-8acb-4bc370c35388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:56.185697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73451ed7-7cda-40b6-8acb-4bc370c35388 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:56.203970 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 829/1662] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:33:56.205027 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd758b6-6725-41c9-8f21-a48e6bdc7075 service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:56.208601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6cd758b6-6725-41c9-8f21-a48e6bdc7075 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:56.229458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:56.273537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 834/1663] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:33:56 2022] POST /identity/v3/projects => generated 403 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:33:56.280631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73451ed7-7cda-40b6-8acb-4bc370c35388 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:56.376948 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd758b6-6725-41c9-8f21-a48e6bdc7075 service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:56.456820 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73451ed7-7cda-40b6-8acb-4bc370c35388 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:56.465175 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a44de-8a9c-4af8-ac0b-3b7a4a4385ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:56.465908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a44de-8a9c-4af8-ac0b-3b7a4a4385ad admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:56.466612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a44de-8a9c-4af8-ac0b-3b7a4a4385ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:56.467469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a44de-8a9c-4af8-ac0b-3b7a4a4385ad admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:56.468204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a44de-8a9c-4af8-ac0b-3b7a4a4385ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:56.470654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a44de-8a9c-4af8-ac0b-3b7a4a4385ad admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:56.482058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:56.486091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa281e-a610-4680-8437-69c89694bb8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:56.487751 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa281e-a610-4680-8437-69c89694bb8d service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:56.488190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa281e-a610-4680-8437-69c89694bb8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:56.489325 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa281e-a610-4680-8437-69c89694bb8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:56.492054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1cfa281e-a610-4680-8437-69c89694bb8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:56.514636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 835/1664] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:33:56 2022] POST /identity/v3/users => generated 302 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:33:56.522889 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a44de-8a9c-4af8-ac0b-3b7a4a4385ad admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:56.555408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a44de-8a9c-4af8-ac0b-3b7a4a4385ad admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:56.573141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3d170-d622-4224-b33a-ec4f30dfda49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:56.573857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3d170-d622-4224-b33a-ec4f30dfda49 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:56.577473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3d170-d622-4224-b33a-ec4f30dfda49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:56.578510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3d170-d622-4224-b33a-ec4f30dfda49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:56.582519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3d170-d622-4224-b33a-ec4f30dfda49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:56.585722 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3d170-d622-4224-b33a-ec4f30dfda49 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:56.605265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa281e-a610-4680-8437-69c89694bb8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:56.606922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa281e-a610-4680-8437-69c89694bb8d service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:56.616130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 836/1665] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:33:56.620304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 830/1666] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:56.621133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3d170-d622-4224-b33a-ec4f30dfda49 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:56.648378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3d170-d622-4224-b33a-ec4f30dfda49 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:56.662774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c61393-130b-48be-85db-70825c9e8dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:56.663644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c61393-130b-48be-85db-70825c9e8dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:56.664002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c61393-130b-48be-85db-70825c9e8dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:56.664289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c61393-130b-48be-85db-70825c9e8dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:56.665101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c61393-130b-48be-85db-70825c9e8dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:56.671019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c61393-130b-48be-85db-70825c9e8dcc admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:56.708550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 837/1667] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:33:56.712857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa281e-a610-4680-8437-69c89694bb8d service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:56.736271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa281e-a610-4680-8437-69c89694bb8d service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:56.757028 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179304bf-2700-4aad-83a7-9f3f10d2bd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:56.759674 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179304bf-2700-4aad-83a7-9f3f10d2bd96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:56.762282 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179304bf-2700-4aad-83a7-9f3f10d2bd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:56.762844 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179304bf-2700-4aad-83a7-9f3f10d2bd96 admin admin] PATH_INFO: `/v3/projects/7923f53751164d94849aefc2ae7744b0/users/d315d995684c4d7ca504a1ea7d55d61d/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:56.817189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179304bf-2700-4aad-83a7-9f3f10d2bd96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7923f53751164d94849aefc2ae7744b0, user_id=d315d995684c4d7ca504a1ea7d55d61d, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:56.824897 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179304bf-2700-4aad-83a7-9f3f10d2bd96 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:56.871253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 831/1668] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:33:56 2022] PUT /identity/v3/projects/7923f53751164d94849aefc2ae7744b0/users/d315d995684c4d7ca504a1ea7d55d61d/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:56.880293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f545f-8d80-4bc3-b05d-b07a9c093395 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:56.880557 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f545f-8d80-4bc3-b05d-b07a9c093395 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:56.880813 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f545f-8d80-4bc3-b05d-b07a9c093395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:56.932247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36837fa-5c8a-4c5b-8232-c2ffc7e404b0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:56.933128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36837fa-5c8a-4c5b-8232-c2ffc7e404b0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:56.933496 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36837fa-5c8a-4c5b-8232-c2ffc7e404b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:56.972897 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-8e8f545f-8d80-4bc3-b05d-b07a9c093395 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:56.979725 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8e8f545f-8d80-4bc3-b05d-b07a9c093395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:57.077880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e36837fa-5c8a-4c5b-8232-c2ffc7e404b0 None None] MFA Rules not processed for user `c43bf6734c984c6e9d189660f903958b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:57.079906 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e36837fa-5c8a-4c5b-8232-c2ffc7e404b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:57.249770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:57.285178 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 838/1669] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:57.291963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f545f-8d80-4bc3-b05d-b07a9c093395 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:57.295367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8e8f545f-8d80-4bc3-b05d-b07a9c093395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:57.332634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f545f-8d80-4bc3-b05d-b07a9c093395 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:57.355953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90a33a-a5dc-4e84-9aa3-441a1d8704b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:57.358857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90a33a-a5dc-4e84-9aa3-441a1d8704b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:57.360518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90a33a-a5dc-4e84-9aa3-441a1d8704b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:57.361583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90a33a-a5dc-4e84-9aa3-441a1d8704b3 admin admin] PATH_INFO: `/v3/users/5621cb413101448c926b37c6519122ac` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:57.365502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90a33a-a5dc-4e84-9aa3-441a1d8704b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5621cb413101448c926b37c6519122ac)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:57.367524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90a33a-a5dc-4e84-9aa3-441a1d8704b3 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:57.374404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 832/1670] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:33:57.377180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36837fa-5c8a-4c5b-8232-c2ffc7e404b0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:57.385350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e36837fa-5c8a-4c5b-8232-c2ffc7e404b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:57.536516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5a90a33a-a5dc-4e84-9aa3-441a1d8704b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5621cb413101448c926b37c6519122ac'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.553511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5a90a33a-a5dc-4e84-9aa3-441a1d8704b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5621cb413101448c926b37c6519122ac'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.554504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5a90a33a-a5dc-4e84-9aa3-441a1d8704b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5621cb413101448c926b37c6519122ac'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.566664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5a90a33a-a5dc-4e84-9aa3-441a1d8704b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5621cb413101448c926b37c6519122ac'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.567859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5a90a33a-a5dc-4e84-9aa3-441a1d8704b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5621cb413101448c926b37c6519122ac'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:33:57.617523 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 839/1671] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:33:57 2022] DELETE /identity/v3/users/5621cb413101448c926b37c6519122ac => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:33:57.619535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca6b80-3a9b-4da8-b77d-dd7e2e32bdd9 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:57.620178 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca6b80-3a9b-4da8-b77d-dd7e2e32bdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:57.621156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca6b80-3a9b-4da8-b77d-dd7e2e32bdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:57.621519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36837fa-5c8a-4c5b-8232-c2ffc7e404b0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:57.683973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-9eca6b80-3a9b-4da8-b77d-dd7e2e32bdd9 None None] MFA Rules not processed for user `c43bf6734c984c6e9d189660f903958b`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:57.687446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9eca6b80-3a9b-4da8-b77d-dd7e2e32bdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:57.781848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af495fc-9077-4790-af77-9c05f44f9126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:57.782829 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af495fc-9077-4790-af77-9c05f44f9126 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:57.784089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af495fc-9077-4790-af77-9c05f44f9126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:57.786079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af495fc-9077-4790-af77-9c05f44f9126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:57.789338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5af495fc-9077-4790-af77-9c05f44f9126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:57.824746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:57.885456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 840/1672] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:57.885456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f702fb-068a-423b-83cb-9faed1aa3cce None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:57.885456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f702fb-068a-423b-83cb-9faed1aa3cce None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:57.885456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f702fb-068a-423b-83cb-9faed1aa3cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:57.914644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-91f702fb-068a-423b-83cb-9faed1aa3cce None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:33:57.917275 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-91f702fb-068a-423b-83cb-9faed1aa3cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:58.011727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af495fc-9077-4790-af77-9c05f44f9126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:58.014551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af495fc-9077-4790-af77-9c05f44f9126 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:58.047429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 833/1673] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:58.048273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af495fc-9077-4790-af77-9c05f44f9126 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:58.058594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5af495fc-9077-4790-af77-9c05f44f9126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:58.088306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 841/1674] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:33:58.121210 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:33:58.170716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f702fb-068a-423b-83cb-9faed1aa3cce None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:33:58.173947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-91f702fb-068a-423b-83cb-9faed1aa3cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:58.208641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f702fb-068a-423b-83cb-9faed1aa3cce None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:58.213643 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af495fc-9077-4790-af77-9c05f44f9126 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:33:58.220189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcdbd5-5c17-4f5d-afcd-1ab5e369c757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:58.222050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcdbd5-5c17-4f5d-afcd-1ab5e369c757 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:58.222901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcdbd5-5c17-4f5d-afcd-1ab5e369c757 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:58.223802 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcdbd5-5c17-4f5d-afcd-1ab5e369c757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:58.233431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191bd7e-00d6-4432-8a57-a921f3123958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:33:58.234129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191bd7e-00d6-4432-8a57-a921f3123958 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:58.234425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191bd7e-00d6-4432-8a57-a921f3123958 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:58.234710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191bd7e-00d6-4432-8a57-a921f3123958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:58.237130 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6191bd7e-00d6-4432-8a57-a921f3123958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:33:58.246111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcdbd5-5c17-4f5d-afcd-1ab5e369c757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:58.250440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcdbd5-5c17-4f5d-afcd-1ab5e369c757 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:58.269000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191bd7e-00d6-4432-8a57-a921f3123958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:33:58.273320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191bd7e-00d6-4432-8a57-a921f3123958 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:33:58.284665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 842/1675] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:33:58.300884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 834/1676] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:33:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:00.640305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaac3a0-c760-403a-aa75-b87fd039b60c None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:00.641052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaac3a0-c760-403a-aa75-b87fd039b60c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:00.641573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaac3a0-c760-403a-aa75-b87fd039b60c None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:00.645448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 843/1677] 10.222.0.42 () {54 vars in 912 bytes} [Thu Jul 21 22:34:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:00.657503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2da455-51f5-4848-88af-1fbabc597f79 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:00.658951 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2da455-51f5-4848-88af-1fbabc597f79 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:00.659647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2da455-51f5-4848-88af-1fbabc597f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:00.837497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7a2da455-51f5-4848-88af-1fbabc597f79 None None] MFA Rules not processed for user `8a120d9e7e204dc5b972fd9402897b41`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:00.839965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7a2da455-51f5-4848-88af-1fbabc597f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:00.863959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 835/1678] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 22:34:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:00.910366 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaac3a0-c760-403a-aa75-b87fd039b60c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:00.917682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-baaac3a0-c760-403a-aa75-b87fd039b60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:00.994144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaac3a0-c760-403a-aa75-b87fd039b60c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:01.017418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73735c-0684-47f4-9b50-d3b89899f4f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:01.018111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73735c-0684-47f4-9b50-d3b89899f4f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:01.018399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73735c-0684-47f4-9b50-d3b89899f4f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:01.018684 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73735c-0684-47f4-9b50-d3b89899f4f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:01.034484 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2da455-51f5-4848-88af-1fbabc597f79 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:01.039526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7a2da455-51f5-4848-88af-1fbabc597f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:01.045990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73735c-0684-47f4-9b50-d3b89899f4f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:01.047485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73735c-0684-47f4-9b50-d3b89899f4f0 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:01.078691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 844/1679] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:01.144632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2da455-51f5-4848-88af-1fbabc597f79 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:01.189503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311ea0e-162b-47e1-94c1-79b634ebd6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:01.191419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311ea0e-162b-47e1-94c1-79b634ebd6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:01.192904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311ea0e-162b-47e1-94c1-79b634ebd6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:01.195514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311ea0e-162b-47e1-94c1-79b634ebd6f7 admin admin] PATH_INFO: `/v3/projects/0e7ddc919f5f4aa2a84749f34d5550e1` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:01.201742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311ea0e-162b-47e1-94c1-79b634ebd6f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e7ddc919f5f4aa2a84749f34d5550e1)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:01.206426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311ea0e-162b-47e1-94c1-79b634ebd6f7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:01.359679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5311ea0e-162b-47e1-94c1-79b634ebd6f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e7ddc919f5f4aa2a84749f34d5550e1', 'request_id': 'req-5311ea0e-162b-47e1-94c1-79b634ebd6f7'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:01.377146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5311ea0e-162b-47e1-94c1-79b634ebd6f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e7ddc919f5f4aa2a84749f34d5550e1', 'request_id': 'req-5311ea0e-162b-47e1-94c1-79b634ebd6f7'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:01.399916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5311ea0e-162b-47e1-94c1-79b634ebd6f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e7ddc919f5f4aa2a84749f34d5550e1', 'request_id': 'req-5311ea0e-162b-47e1-94c1-79b634ebd6f7'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:01.426085 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5311ea0e-162b-47e1-94c1-79b634ebd6f7 admin admin] The token cache is being invalidate because project 0e7ddc919f5f4aa2a84749f34d5550e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:01.433240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5311ea0e-162b-47e1-94c1-79b634ebd6f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:01.435255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5311ea0e-162b-47e1-94c1-79b634ebd6f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:01.442555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 836/1680] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:01 2022] DELETE /identity/v3/projects/0e7ddc919f5f4aa2a84749f34d5550e1 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:01.446910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3603e-8195-4ce8-b2f1-e18e8cec4ec6 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:01.447519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3603e-8195-4ce8-b2f1-e18e8cec4ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:01.447948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3603e-8195-4ce8-b2f1-e18e8cec4ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:01.521395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-dde3603e-8195-4ce8-b2f1-e18e8cec4ec6 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:01.524969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-dde3603e-8195-4ce8-b2f1-e18e8cec4ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:01.816897 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 845/1681] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:01.826607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d313ddf-a0d3-44f9-9b51-0d9d3c2018af None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:01.827164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d313ddf-a0d3-44f9-9b51-0d9d3c2018af None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:01.827649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d313ddf-a0d3-44f9-9b51-0d9d3c2018af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:01.867511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2d313ddf-a0d3-44f9-9b51-0d9d3c2018af None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:01.868821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2d313ddf-a0d3-44f9-9b51-0d9d3c2018af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:01.917522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 837/1682] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:01.921853 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3603e-8195-4ce8-b2f1-e18e8cec4ec6 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:01.943245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3603e-8195-4ce8-b2f1-e18e8cec4ec6 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:01.957753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091871f6-0a98-4e38-afa9-9f31c10efcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:01.958697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091871f6-0a98-4e38-afa9-9f31c10efcc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:01.962760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091871f6-0a98-4e38-afa9-9f31c10efcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:01.967794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091871f6-0a98-4e38-afa9-9f31c10efcc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:01.968955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091871f6-0a98-4e38-afa9-9f31c10efcc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:01.974743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091871f6-0a98-4e38-afa9-9f31c10efcc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:01.976154 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091871f6-0a98-4e38-afa9-9f31c10efcc5 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:02.054926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 846/1683] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:02.055434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d313ddf-a0d3-44f9-9b51-0d9d3c2018af None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:02.067510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d313ddf-a0d3-44f9-9b51-0d9d3c2018af None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:02.078264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7efce-3d3e-4a9d-8f1e-525360cee1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:02.079665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7efce-3d3e-4a9d-8f1e-525360cee1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.080141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7efce-3d3e-4a9d-8f1e-525360cee1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.080588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7efce-3d3e-4a9d-8f1e-525360cee1c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.084181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7efce-3d3e-4a9d-8f1e-525360cee1c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:02.086544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7efce-3d3e-4a9d-8f1e-525360cee1c9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:02.146436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 838/1684] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:02 2022] POST /identity/v3/projects => generated 363 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:02.150457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091871f6-0a98-4e38-afa9-9f31c10efcc5 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:02.197031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091871f6-0a98-4e38-afa9-9f31c10efcc5 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:02.343196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:02.377320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380b94f-4386-40dd-96be-a6e37d4a9fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:02.382413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380b94f-4386-40dd-96be-a6e37d4a9fb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.382871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380b94f-4386-40dd-96be-a6e37d4a9fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.383429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380b94f-4386-40dd-96be-a6e37d4a9fb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.384822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380b94f-4386-40dd-96be-a6e37d4a9fb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:02.389457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380b94f-4386-40dd-96be-a6e37d4a9fb2 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:02.451112 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7efce-3d3e-4a9d-8f1e-525360cee1c9 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:02.456909 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 847/1685] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:02 2022] POST /identity/v3/users => generated 282 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:02.476462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7efce-3d3e-4a9d-8f1e-525360cee1c9 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:02.487694 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7b356-c0d2-4cc5-a8cd-aa6181ef6351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:02.488425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7b356-c0d2-4cc5-a8cd-aa6181ef6351 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.488740 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7b356-c0d2-4cc5-a8cd-aa6181ef6351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.489020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7b356-c0d2-4cc5-a8cd-aa6181ef6351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.489791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec7b356-c0d2-4cc5-a8cd-aa6181ef6351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:02.491504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec7b356-c0d2-4cc5-a8cd-aa6181ef6351 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:02.520200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 839/1686] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:02.521483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380b94f-4386-40dd-96be-a6e37d4a9fb2 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:02.542362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380b94f-4386-40dd-96be-a6e37d4a9fb2 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:02.557662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab9718-1a2c-4ac2-8c02-9774b74a148e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:02.558570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab9718-1a2c-4ac2-8c02-9774b74a148e admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.559020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab9718-1a2c-4ac2-8c02-9774b74a148e admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.559447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab9718-1a2c-4ac2-8c02-9774b74a148e admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.561091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab9718-1a2c-4ac2-8c02-9774b74a148e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:02.572708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab9718-1a2c-4ac2-8c02-9774b74a148e admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:02.597655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 848/1687] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:02.602477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7b356-c0d2-4cc5-a8cd-aa6181ef6351 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:02.625944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7b356-c0d2-4cc5-a8cd-aa6181ef6351 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:02.641343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168903a-7659-4f7e-b103-bffcbfd600b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:02.642297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168903a-7659-4f7e-b103-bffcbfd600b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.642583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168903a-7659-4f7e-b103-bffcbfd600b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.642863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168903a-7659-4f7e-b103-bffcbfd600b1 admin admin] PATH_INFO: `/v3/projects/67a4f7879e6e46dd9683eeda372d07bf/users/98b0cb05204c4867a73ac4d1180ed005/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.681244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168903a-7659-4f7e-b103-bffcbfd600b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67a4f7879e6e46dd9683eeda372d07bf, user_id=98b0cb05204c4867a73ac4d1180ed005, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:02.684066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168903a-7659-4f7e-b103-bffcbfd600b1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:02.723344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 840/1688] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:02 2022] PUT /identity/v3/projects/67a4f7879e6e46dd9683eeda372d07bf/users/98b0cb05204c4867a73ac4d1180ed005/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:02.724616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a8981-f4fc-44c2-8c11-b178068437d2 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.724920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a8981-f4fc-44c2-8c11-b178068437d2 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.725240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a8981-f4fc-44c2-8c11-b178068437d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.768011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fdef7-dca8-40aa-bcb3-d536cb5c0387 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.768336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fdef7-dca8-40aa-bcb3-d536cb5c0387 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.768813 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fdef7-dca8-40aa-bcb3-d536cb5c0387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:02.786304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-0c2a8981-f4fc-44c2-8c11-b178068437d2 None None] MFA Rules not processed for user `98b0cb05204c4867a73ac4d1180ed005`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:02.788030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0c2a8981-f4fc-44c2-8c11-b178068437d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:02.811134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6d4fdef7-dca8-40aa-bcb3-d536cb5c0387 None None] MFA Rules not processed for user `f93bd44f2fbc4be4be905c8b8ae0e343`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:02.813026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6d4fdef7-dca8-40aa-bcb3-d536cb5c0387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:02.831073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:02.892014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 849/1689] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:02.900768 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a8981-f4fc-44c2-8c11-b178068437d2 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:02.971969 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 841/1690] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:02.983889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac370200-1f97-41f8-85ab-dcd636dde36e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:02.984417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac370200-1f97-41f8-85ab-dcd636dde36e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:02.984885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac370200-1f97-41f8-85ab-dcd636dde36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.026055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ac370200-1f97-41f8-85ab-dcd636dde36e None None] MFA Rules not processed for user `f93bd44f2fbc4be4be905c8b8ae0e343`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:03.027632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ac370200-1f97-41f8-85ab-dcd636dde36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:03.040198 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a8981-f4fc-44c2-8c11-b178068437d2 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:03.050238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 842/1691] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:03.058585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8deb17-35c2-4eeb-889d-c29cb4984f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:03.059673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8deb17-35c2-4eeb-889d-c29cb4984f2d admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.060183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8deb17-35c2-4eeb-889d-c29cb4984f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.061283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8deb17-35c2-4eeb-889d-c29cb4984f2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.062010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d890c55-11d7-4b84-af9f-027f0b69e9cf None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.062456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d890c55-11d7-4b84-af9f-027f0b69e9cf None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.062802 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8deb17-35c2-4eeb-889d-c29cb4984f2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:03.063087 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d890c55-11d7-4b84-af9f-027f0b69e9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.071852 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8deb17-35c2-4eeb-889d-c29cb4984f2d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:03.113583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4d890c55-11d7-4b84-af9f-027f0b69e9cf None None] MFA Rules not processed for user `da46b59a0cb5430589aeef1693b2ce50`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:03.115071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4d890c55-11d7-4b84-af9f-027f0b69e9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:03.144842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 850/1692] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:02 2022] POST /identity/v3/projects => generated 365 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:03.146015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8deb17-35c2-4eeb-889d-c29cb4984f2d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:03.161459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-eb8deb17-35c2-4eeb-889d-c29cb4984f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:03.259915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 843/1693] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:03.259915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac85a8-f469-4230-9302-3e61a9a64383 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.259915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac85a8-f469-4230-9302-3e61a9a64383 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.259915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac85a8-f469-4230-9302-3e61a9a64383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.270713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:03.352983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8deb17-35c2-4eeb-889d-c29cb4984f2d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:03.375924 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05435b-379a-4c44-be7f-b569c2c7d2aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:03.376663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05435b-379a-4c44-be7f-b569c2c7d2aa service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.377126 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05435b-379a-4c44-be7f-b569c2c7d2aa service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.377488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05435b-379a-4c44-be7f-b569c2c7d2aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.453287 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0eac85a8-f469-4230-9302-3e61a9a64383 None None] MFA Rules not processed for user `d315d995684c4d7ca504a1ea7d55d61d`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:03.459779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0eac85a8-f469-4230-9302-3e61a9a64383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:03.566338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05435b-379a-4c44-be7f-b569c2c7d2aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:03.568674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05435b-379a-4c44-be7f-b569c2c7d2aa service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:03.589057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 851/1694] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:03.590400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05435b-379a-4c44-be7f-b569c2c7d2aa service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:03.797227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05435b-379a-4c44-be7f-b569c2c7d2aa service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:03.807578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489805b6-e413-49bc-bcfc-1ae09927a686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:03.808306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489805b6-e413-49bc-bcfc-1ae09927a686 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.808609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489805b6-e413-49bc-bcfc-1ae09927a686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.808932 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489805b6-e413-49bc-bcfc-1ae09927a686 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.809663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489805b6-e413-49bc-bcfc-1ae09927a686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:03.812848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489805b6-e413-49bc-bcfc-1ae09927a686 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:03.834943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 844/1695] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 582 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:03.857562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf254d-3988-4757-b32d-1755dc9cde37 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.858792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf254d-3988-4757-b32d-1755dc9cde37 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.859327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf254d-3988-4757-b32d-1755dc9cde37 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.861334 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 852/1696] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:03 2022] POST /identity/v3/users => generated 289 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:03.862334 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489805b6-e413-49bc-bcfc-1ae09927a686 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:03.867585 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 845/1697] 10.222.0.42 () {54 vars in 961 bytes} [Thu Jul 21 22:34:03 2022] GET /identity => generated 270 bytes in 32 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:03.876388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-489805b6-e413-49bc-bcfc-1ae09927a686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:03.878236 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf254d-3988-4757-b32d-1755dc9cde37 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:03.910526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489805b6-e413-49bc-bcfc-1ae09927a686 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:03.933549 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704f1d1-a81a-48ae-a797-05cb496ef9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:03.934538 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704f1d1-a81a-48ae-a797-05cb496ef9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.934965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704f1d1-a81a-48ae-a797-05cb496ef9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.935477 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704f1d1-a81a-48ae-a797-05cb496ef9a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.936425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf254d-3988-4757-b32d-1755dc9cde37 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:03.936596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704f1d1-a81a-48ae-a797-05cb496ef9a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:03.938806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704f1d1-a81a-48ae-a797-05cb496ef9a3 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:03.951838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f07d4-9c7d-4dec-9c63-8624a1c1c40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:03.953423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f07d4-9c7d-4dec-9c63-8624a1c1c40a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:03.954480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f07d4-9c7d-4dec-9c63-8624a1c1c40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:03.955384 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f07d4-9c7d-4dec-9c63-8624a1c1c40a admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:03.957857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f07d4-9c7d-4dec-9c63-8624a1c1c40a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:03.972897 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f07d4-9c7d-4dec-9c63-8624a1c1c40a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:04.002020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 853/1698] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:03 2022] POST /identity/v3/projects => generated 403 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:04.003630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda11a48-905e-4b85-a2c7-b54bb54312ef None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.005062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda11a48-905e-4b85-a2c7-b54bb54312ef None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.005545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda11a48-905e-4b85-a2c7-b54bb54312ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.014371 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 846/1699] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:04.027465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f07d4-9c7d-4dec-9c63-8624a1c1c40a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:04.057390 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f07d4-9c7d-4dec-9c63-8624a1c1c40a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:04.067256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-dda11a48-905e-4b85-a2c7-b54bb54312ef None None] MFA Rules not processed for user `705ff0e73e95499e873b1df2a24fefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:04.069488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-dda11a48-905e-4b85-a2c7-b54bb54312ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:04.141862 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:04.223057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d38cb8-3c54-43b2-a220-eea5f0f93d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:04.225830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d38cb8-3c54-43b2-a220-eea5f0f93d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.226247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d38cb8-3c54-43b2-a220-eea5f0f93d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.226573 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d38cb8-3c54-43b2-a220-eea5f0f93d2f admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.228935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d38cb8-3c54-43b2-a220-eea5f0f93d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:04.234614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d38cb8-3c54-43b2-a220-eea5f0f93d2f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:04.262611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:04.288527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 847/1700] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:04 2022] POST /identity/v3/users => generated 308 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:04.288527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d38cb8-3c54-43b2-a220-eea5f0f93d2f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:04.337311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d38cb8-3c54-43b2-a220-eea5f0f93d2f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:04.367955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7ce71-d830-4631-bb25-6810ae9946d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:04.369786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ce71-d830-4631-bb25-6810ae9946d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.370156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ce71-d830-4631-bb25-6810ae9946d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.373102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7ce71-d830-4631-bb25-6810ae9946d8 admin admin] PATH_INFO: `/v3/projects/91f4cd559ae5435c852efe424bf028ca/users/245fae7b529c455d9e441004faafe26b/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.409321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 854/1701] 10.222.0.42 () {58 vars in 1058 bytes} [Thu Jul 21 22:34:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:04.411926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda11a48-905e-4b85-a2c7-b54bb54312ef None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:04.454368 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7ce71-d830-4631-bb25-6810ae9946d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f4cd559ae5435c852efe424bf028ca, user_id=245fae7b529c455d9e441004faafe26b, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:04.457254 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7ce71-d830-4631-bb25-6810ae9946d8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:04.473447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda11a48-905e-4b85-a2c7-b54bb54312ef None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:04.494248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85034dc1-8ebd-460d-8c93-5261fe82616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:04.495850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 848/1702] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:04 2022] PUT /identity/v3/projects/91f4cd559ae5435c852efe424bf028ca/users/245fae7b529c455d9e441004faafe26b/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:04.496746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7ce71-d830-4631-bb25-6810ae9946d8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:04.503954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85034dc1-8ebd-460d-8c93-5261fe82616c admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.505058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85034dc1-8ebd-460d-8c93-5261fe82616c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.505609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85034dc1-8ebd-460d-8c93-5261fe82616c admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.506942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85034dc1-8ebd-460d-8c93-5261fe82616c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:04.524761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85034dc1-8ebd-460d-8c93-5261fe82616c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:04.575492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 855/1703] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:04.583118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85034dc1-8ebd-460d-8c93-5261fe82616c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:04.635379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85034dc1-8ebd-460d-8c93-5261fe82616c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:04.736506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7ce71-d830-4631-bb25-6810ae9946d8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:04.756107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdeb8a-a236-49e7-abcc-bbd63d5f0cb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:04.756834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdeb8a-a236-49e7-abcc-bbd63d5f0cb2 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.757442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdeb8a-a236-49e7-abcc-bbd63d5f0cb2 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.757442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdeb8a-a236-49e7-abcc-bbd63d5f0cb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.766988 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-31bdeb8a-a236-49e7-abcc-bbd63d5f0cb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:04.798245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdeb8a-a236-49e7-abcc-bbd63d5f0cb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:04.805293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bdeb8a-a236-49e7-abcc-bbd63d5f0cb2 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:04.839821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 849/1704] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:04.854855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdeb8a-a236-49e7-abcc-bbd63d5f0cb2 service placement] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:05.035200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e6d5f-265d-4042-a00e-648bf5de2a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:05.035954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e6d5f-265d-4042-a00e-648bf5de2a29 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.036210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e6d5f-265d-4042-a00e-648bf5de2a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.036518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e6d5f-265d-4042-a00e-648bf5de2a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.046849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e6d5f-265d-4042-a00e-648bf5de2a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:05.048692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e6d5f-265d-4042-a00e-648bf5de2a29 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:05.098097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 856/1705] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 522 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:05.102530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e86dd-d717-4d25-82b5-ba7a598d9234 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.108225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e86dd-d717-4d25-82b5-ba7a598d9234 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.108225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e86dd-d717-4d25-82b5-ba7a598d9234 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.108225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 857/1706] 10.222.0.42 () {54 vars in 912 bytes} [Thu Jul 21 22:34:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:05.119929 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e86dd-d717-4d25-82b5-ba7a598d9234 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:05.176456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e86dd-d717-4d25-82b5-ba7a598d9234 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:05.190230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdeb8a-a236-49e7-abcc-bbd63d5f0cb2 service placement] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:05.209895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8907920-1cdd-4815-ab8e-b52f90d3a5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:05.211983 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8907920-1cdd-4815-ab8e-b52f90d3a5ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.212503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8907920-1cdd-4815-ab8e-b52f90d3a5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.214316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8907920-1cdd-4815-ab8e-b52f90d3a5ce admin admin] PATH_INFO: `/v3/projects/90eda2a6e3fc44ef952e2bdd61f0b27d/users/ef25766307494bee9f326aa44d074c48/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.218233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9f58c-fc97-4383-9410-dcbea2ad5ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:05.225976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9f58c-fc97-4383-9410-dcbea2ad5ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.226786 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9f58c-fc97-4383-9410-dcbea2ad5ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.227468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9f58c-fc97-4383-9410-dcbea2ad5ef2 admin admin] PATH_INFO: `/v3/projects/91f4cd559ae5435c852efe424bf028ca/users/245fae7b529c455d9e441004faafe26b/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.239937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9f58c-fc97-4383-9410-dcbea2ad5ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f4cd559ae5435c852efe424bf028ca, user_id=245fae7b529c455d9e441004faafe26b, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:05.271697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9f58c-fc97-4383-9410-dcbea2ad5ef2 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:05.311544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8907920-1cdd-4815-ab8e-b52f90d3a5ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=90eda2a6e3fc44ef952e2bdd61f0b27d, user_id=ef25766307494bee9f326aa44d074c48, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:05.319415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8907920-1cdd-4815-ab8e-b52f90d3a5ce admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:05.351108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 858/1707] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:05 2022] PUT /identity/v3/projects/91f4cd559ae5435c852efe424bf028ca/users/245fae7b529c455d9e441004faafe26b/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:05.356231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec29a7e-9fbe-4fcd-8d24-c507b63c0d98 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.358570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec29a7e-9fbe-4fcd-8d24-c507b63c0d98 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.358570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec29a7e-9fbe-4fcd-8d24-c507b63c0d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.416740 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 850/1708] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:04 2022] PUT /identity/v3/projects/90eda2a6e3fc44ef952e2bdd61f0b27d/users/ef25766307494bee9f326aa44d074c48/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:05.426079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ad081-5f53-4493-a269-088a2962c57b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.426378 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ad081-5f53-4493-a269-088a2962c57b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.426665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ad081-5f53-4493-a269-088a2962c57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.511464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-5ec29a7e-9fbe-4fcd-8d24-c507b63c0d98 None None] MFA Rules not processed for user `8a120d9e7e204dc5b972fd9402897b41`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:05.524732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5ec29a7e-9fbe-4fcd-8d24-c507b63c0d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:05.560812 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:05.601338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-319ad081-5f53-4493-a269-088a2962c57b None None] MFA Rules not processed for user `245fae7b529c455d9e441004faafe26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:05.610137 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-319ad081-5f53-4493-a269-088a2962c57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:05.702985 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 859/1709] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 22:34:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:05.704406 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec29a7e-9fbe-4fcd-8d24-c507b63c0d98 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:05.890014 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 851/1710] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:05.893525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ad081-5f53-4493-a269-088a2962c57b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:05.897635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-319ad081-5f53-4493-a269-088a2962c57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:05.953086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ad081-5f53-4493-a269-088a2962c57b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:05.983396 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237b06d-20fd-451c-a5d2-77c95e5da487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:05.984621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237b06d-20fd-451c-a5d2-77c95e5da487 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.985135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237b06d-20fd-451c-a5d2-77c95e5da487 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.985615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237b06d-20fd-451c-a5d2-77c95e5da487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:06.016111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec29a7e-9fbe-4fcd-8d24-c507b63c0d98 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:06.022632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237b06d-20fd-451c-a5d2-77c95e5da487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:06.029416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237b06d-20fd-451c-a5d2-77c95e5da487 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:06.040955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56927f11-393c-4978-b40c-fda3732845cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:06.040955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927f11-393c-4978-b40c-fda3732845cb admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:06.041681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927f11-393c-4978-b40c-fda3732845cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:06.041853 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927f11-393c-4978-b40c-fda3732845cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:06.043300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56927f11-393c-4978-b40c-fda3732845cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:06.049911 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56927f11-393c-4978-b40c-fda3732845cb admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:06.079010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 852/1711] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:06.084269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237b06d-20fd-451c-a5d2-77c95e5da487 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:06.084576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:06.090569 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3237b06d-20fd-451c-a5d2-77c95e5da487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:06.110661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 860/1712] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:06.116230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56927f11-393c-4978-b40c-fda3732845cb admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:06.147391 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56927f11-393c-4978-b40c-fda3732845cb admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:06.165574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d774e6f-935a-4608-933a-5bb5b753b734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:06.166936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d774e6f-935a-4608-933a-5bb5b753b734 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:06.167392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d774e6f-935a-4608-933a-5bb5b753b734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:06.167822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d774e6f-935a-4608-933a-5bb5b753b734 admin admin] PATH_INFO: `/v3/projects/90eda2a6e3fc44ef952e2bdd61f0b27d/users/ef25766307494bee9f326aa44d074c48/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:06.174180 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d774e6f-935a-4608-933a-5bb5b753b734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90eda2a6e3fc44ef952e2bdd61f0b27d, user_id=ef25766307494bee9f326aa44d074c48, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:06.191460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d774e6f-935a-4608-933a-5bb5b753b734 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:06.264318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 861/1713] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:06 2022] PUT /identity/v3/projects/90eda2a6e3fc44ef952e2bdd61f0b27d/users/ef25766307494bee9f326aa44d074c48/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:06.399452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237b06d-20fd-451c-a5d2-77c95e5da487 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:06.641010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecb6c9-aedc-4438-9fe3-94f85407d39e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:06.641990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecb6c9-aedc-4438-9fe3-94f85407d39e service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:06.645726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecb6c9-aedc-4438-9fe3-94f85407d39e service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:06.646167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecb6c9-aedc-4438-9fe3-94f85407d39e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:06.978189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:07.054443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecb6c9-aedc-4438-9fe3-94f85407d39e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:07.063373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecb6c9-aedc-4438-9fe3-94f85407d39e service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:07.159276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 853/1714] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 1080 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:07.421515 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d774e6f-935a-4608-933a-5bb5b753b734 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:07.425063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5d774e6f-935a-4608-933a-5bb5b753b734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:07.506037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d774e6f-935a-4608-933a-5bb5b753b734 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:07.515896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68313932-328b-46a6-9d1a-a236084d06cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:07.519678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68313932-328b-46a6-9d1a-a236084d06cc service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.520563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68313932-328b-46a6-9d1a-a236084d06cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.521333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68313932-328b-46a6-9d1a-a236084d06cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.718493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68313932-328b-46a6-9d1a-a236084d06cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:07.724731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68313932-328b-46a6-9d1a-a236084d06cc service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:07.745194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 862/1715] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:07 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:08.170791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecb6c9-aedc-4438-9fe3-94f85407d39e service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:08.170791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-66ecb6c9-aedc-4438-9fe3-94f85407d39e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:08.179288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fb413-29ea-4341-b052-dfa0a2bff549 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:08.180137 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fb413-29ea-4341-b052-dfa0a2bff549 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:08.180960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fb413-29ea-4341-b052-dfa0a2bff549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:08.265400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ac5fb413-29ea-4341-b052-dfa0a2bff549 None None] MFA Rules not processed for user `f93bd44f2fbc4be4be905c8b8ae0e343`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:08.277990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ac5fb413-29ea-4341-b052-dfa0a2bff549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:08.578890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecb6c9-aedc-4438-9fe3-94f85407d39e service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:08.590473 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:08.595692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:08.596144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:08.596787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] PATH_INFO: `/v3/users/7a798c8bc70c4469972283bf92c39735` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:08.600393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a798c8bc70c4469972283bf92c39735)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:08.605450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:08.625583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 863/1716] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:08.848136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a798c8bc70c4469972283bf92c39735'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.882454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a798c8bc70c4469972283bf92c39735'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.936745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a798c8bc70c4469972283bf92c39735'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.953443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a798c8bc70c4469972283bf92c39735'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.954224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a798c8bc70c4469972283bf92c39735'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:08.967025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 854/1717] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:08 2022] DELETE /identity/v3/users/7a798c8bc70c4469972283bf92c39735 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:08.969772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097cdab-da69-4d6a-a459-7f1319a1a07f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:08.970321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097cdab-da69-4d6a-a459-7f1319a1a07f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:08.970832 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097cdab-da69-4d6a-a459-7f1319a1a07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:09.032677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f097cdab-da69-4d6a-a459-7f1319a1a07f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:09.036419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f097cdab-da69-4d6a-a459-7f1319a1a07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:09.251494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 864/1718] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:09.320194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:09.323080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:09.520523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82105ef1-dff5-4c4f-9656-09cacc61f171 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:09.530539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373ac0b-bca8-4397-9499-623fcd8ba13d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:09.533386 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373ac0b-bca8-4397-9499-623fcd8ba13d service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:09.533560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373ac0b-bca8-4397-9499-623fcd8ba13d service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:09.533844 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373ac0b-bca8-4397-9499-623fcd8ba13d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:09.546442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373ac0b-bca8-4397-9499-623fcd8ba13d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:09.548162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373ac0b-bca8-4397-9499-623fcd8ba13d service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:09.562754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 855/1719] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:09.844870 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f11de3-a4f5-452c-bdb4-a857a5590ea0 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:09.845770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f11de3-a4f5-452c-bdb4-a857a5590ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:09.845770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f11de3-a4f5-452c-bdb4-a857a5590ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:09.940066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-54f11de3-a4f5-452c-bdb4-a857a5590ea0 None None] MFA Rules not processed for user `ef25766307494bee9f326aa44d074c48`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:09.941415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-54f11de3-a4f5-452c-bdb4-a857a5590ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:09.996329 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:10.107080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 865/1720] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:09 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:10.979505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373ac0b-bca8-4397-9499-623fcd8ba13d service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.982912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f373ac0b-bca8-4397-9499-623fcd8ba13d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:11.028349 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373ac0b-bca8-4397-9499-623fcd8ba13d service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:11.055700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:11.056644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:11.056937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:11.057214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] PATH_INFO: `/v3/users/b802fcb48ff9485b8c4dc20f9a0a9a48` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:11.060314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b802fcb48ff9485b8c4dc20f9a0a9a48)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:11.065211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:11.079050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:11.302172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b802fcb48ff9485b8c4dc20f9a0a9a48'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:11.331245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b802fcb48ff9485b8c4dc20f9a0a9a48'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:11.368745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b802fcb48ff9485b8c4dc20f9a0a9a48'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:11.387085 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b802fcb48ff9485b8c4dc20f9a0a9a48'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:11.389408 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b802fcb48ff9485b8c4dc20f9a0a9a48'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:11.410888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 856/1721] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:10 2022] DELETE /identity/v3/users/b802fcb48ff9485b8c4dc20f9a0a9a48 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:11.417492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb9eb0-5dbb-4782-bce5-05e8a9b7334b None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:11.418640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb9eb0-5dbb-4782-bce5-05e8a9b7334b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:11.419283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb9eb0-5dbb-4782-bce5-05e8a9b7334b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:11.490605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-fbeb9eb0-5dbb-4782-bce5-05e8a9b7334b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:11.495354 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fbeb9eb0-5dbb-4782-bce5-05e8a9b7334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:11.777903 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:11.779182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 866/1722] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:11.781070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:11.829465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3d97b-7616-42f4-a050-b6932dc468a2 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:11.876237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:11.880593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:11.881352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:11.881954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] PATH_INFO: `/v3/projects/61132d14ab244e878995e6b7199dd19f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:11.887548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=61132d14ab244e878995e6b7199dd19f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:11.895340 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:11.990799 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb9eb0-5dbb-4782-bce5-05e8a9b7334b None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:11.992470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fbeb9eb0-5dbb-4782-bce5-05e8a9b7334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:12.041544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61132d14ab244e878995e6b7199dd19f', 'request_id': 'req-508a1012-183c-4074-99d4-66fcb79894ca'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.068647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61132d14ab244e878995e6b7199dd19f', 'request_id': 'req-508a1012-183c-4074-99d4-66fcb79894ca'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.080040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61132d14ab244e878995e6b7199dd19f', 'request_id': 'req-508a1012-183c-4074-99d4-66fcb79894ca'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.116498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] The token cache is being invalidate because project 61132d14ab244e878995e6b7199dd19f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:12.117282 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.118301 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.137484 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 857/1723] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:11 2022] DELETE /identity/v3/projects/61132d14ab244e878995e6b7199dd19f => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:12.141551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:12.146716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:12.253013 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb9eb0-5dbb-4782-bce5-05e8a9b7334b None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:12.455615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a1012-183c-4074-99d4-66fcb79894ca admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:12.483363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:12.484461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:12.484776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:12.485056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] PATH_INFO: `/v3/users/3b755eb3682b4686a68a5fcb4bb67266` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:12.487701 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b755eb3682b4686a68a5fcb4bb67266)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:12.496472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:12.516811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:12.610822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48612a87-ab03-47f3-a6ef-24b1d2dbf63c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:12.611633 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48612a87-ab03-47f3-a6ef-24b1d2dbf63c service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:12.611969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48612a87-ab03-47f3-a6ef-24b1d2dbf63c service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:12.617137 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48612a87-ab03-47f3-a6ef-24b1d2dbf63c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:12.621032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-48612a87-ab03-47f3-a6ef-24b1d2dbf63c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:12.677917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48612a87-ab03-47f3-a6ef-24b1d2dbf63c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:12.686061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48612a87-ab03-47f3-a6ef-24b1d2dbf63c service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:12.705852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b755eb3682b4686a68a5fcb4bb67266'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.718197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b755eb3682b4686a68a5fcb4bb67266'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.723055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 867/1724] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 733 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:12.727935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fcfd7-0185-4f1e-8215-bd80ceb7333f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:12.729784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fcfd7-0185-4f1e-8215-bd80ceb7333f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:12.735322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fcfd7-0185-4f1e-8215-bd80ceb7333f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:12.738865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b755eb3682b4686a68a5fcb4bb67266'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.751207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b755eb3682b4686a68a5fcb4bb67266'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.752400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b755eb3682b4686a68a5fcb4bb67266'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:12.772092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 858/1725] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:12 2022] DELETE /identity/v3/users/3b755eb3682b4686a68a5fcb4bb67266 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:12.798095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:12.858411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-eb4fcfd7-0185-4f1e-8215-bd80ceb7333f None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:12.861379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-eb4fcfd7-0185-4f1e-8215-bd80ceb7333f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:13.030539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 868/1726] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 22:34:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:13.230170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:13.233475 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:13.459332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62769051-fc47-4697-814a-930be3b320ef admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:13.465718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c4439-ce74-4882-bad8-fe2a5aca9229 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:13.466947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c4439-ce74-4882-bad8-fe2a5aca9229 service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.467210 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c4439-ce74-4882-bad8-fe2a5aca9229 service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.467489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c4439-ce74-4882-bad8-fe2a5aca9229 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.491385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c4439-ce74-4882-bad8-fe2a5aca9229 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:13.493785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c4439-ce74-4882-bad8-fe2a5aca9229 service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:13.509971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 859/1727] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 22:34:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:34:13.553146 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411eba7e-6ef3-4f2e-accf-e34194e3cfdb None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.553518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411eba7e-6ef3-4f2e-accf-e34194e3cfdb None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.553709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411eba7e-6ef3-4f2e-accf-e34194e3cfdb None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.555313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 869/1728] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 22:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:13.645048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42129f3-6acb-4534-8f24-732956822591 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.645663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42129f3-6acb-4534-8f24-732956822591 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.646122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42129f3-6acb-4534-8f24-732956822591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.707171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b42129f3-6acb-4534-8f24-732956822591 None None] MFA Rules not processed for user `705ff0e73e95499e873b1df2a24fefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:13.711310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b42129f3-6acb-4534-8f24-732956822591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:13.920641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:13.961701 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 860/1729] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 22:34:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:13.990497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411eba7e-6ef3-4f2e-accf-e34194e3cfdb None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:13.993863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-411eba7e-6ef3-4f2e-accf-e34194e3cfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:14.073030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411eba7e-6ef3-4f2e-accf-e34194e3cfdb None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:14.082014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece144a4-7f01-4616-950d-cb51edebccec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:14.083319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece144a4-7f01-4616-950d-cb51edebccec service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:14.083847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece144a4-7f01-4616-950d-cb51edebccec service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:14.084367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece144a4-7f01-4616-950d-cb51edebccec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:14.118048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece144a4-7f01-4616-950d-cb51edebccec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:14.121362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece144a4-7f01-4616-950d-cb51edebccec service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:14.143161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 870/1730] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:14.877484 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42129f3-6acb-4534-8f24-732956822591 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:14.888722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b42129f3-6acb-4534-8f24-732956822591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:14.985153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42129f3-6acb-4534-8f24-732956822591 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:14.996972 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249d919-14b5-446c-828f-62e8c49f53f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:14.998202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249d919-14b5-446c-828f-62e8c49f53f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:14.998488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249d919-14b5-446c-828f-62e8c49f53f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:14.998770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249d919-14b5-446c-828f-62e8c49f53f9 admin admin] PATH_INFO: `/v3/projects/156a0bfceb934d2498348f120b387520` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.001608 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249d919-14b5-446c-828f-62e8c49f53f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=156a0bfceb934d2498348f120b387520)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.003485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249d919-14b5-446c-828f-62e8c49f53f9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.117785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece144a4-7f01-4616-950d-cb51edebccec service placement] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:15.121094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ece144a4-7f01-4616-950d-cb51edebccec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:15.161684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9249d919-14b5-446c-828f-62e8c49f53f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '156a0bfceb934d2498348f120b387520', 'request_id': 'req-9249d919-14b5-446c-828f-62e8c49f53f9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.166256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece144a4-7f01-4616-950d-cb51edebccec service placement] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:15.176568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9249d919-14b5-446c-828f-62e8c49f53f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '156a0bfceb934d2498348f120b387520', 'request_id': 'req-9249d919-14b5-446c-828f-62e8c49f53f9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.197745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9249d919-14b5-446c-828f-62e8c49f53f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '156a0bfceb934d2498348f120b387520', 'request_id': 'req-9249d919-14b5-446c-828f-62e8c49f53f9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.226723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9249d919-14b5-446c-828f-62e8c49f53f9 admin admin] The token cache is being invalidate because project 156a0bfceb934d2498348f120b387520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:15.227082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9249d919-14b5-446c-828f-62e8c49f53f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.229334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9249d919-14b5-446c-828f-62e8c49f53f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.234942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 861/1731] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:14 2022] DELETE /identity/v3/projects/156a0bfceb934d2498348f120b387520 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:15.239963 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de341cd8-52bc-4f66-be46-3a5e3702b8da None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.240681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de341cd8-52bc-4f66-be46-3a5e3702b8da None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.240805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de341cd8-52bc-4f66-be46-3a5e3702b8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.266935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:15.267934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.268212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.268727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59 admin admin] PATH_INFO: `/v3/projects/9e5c871bb5664dba96c9f7692e5285a5` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.272044 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e5c871bb5664dba96c9f7692e5285a5)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.277526 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.302886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-de341cd8-52bc-4f66-be46-3a5e3702b8da None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:15.304651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-de341cd8-52bc-4f66-be46-3a5e3702b8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:15.324398 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 862/1732] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:15.332515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5f257-e0e0-4ed5-8eec-4dc7ee3c2291 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.332776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5f257-e0e0-4ed5-8eec-4dc7ee3c2291 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.333237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5f257-e0e0-4ed5-8eec-4dc7ee3c2291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.358469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-46f5f257-e0e0-4ed5-8eec-4dc7ee3c2291 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:15.359853 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-46f5f257-e0e0-4ed5-8eec-4dc7ee3c2291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:15.382185 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e5c871bb5664dba96c9f7692e5285a5', 'request_id': 'req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.401799 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e5c871bb5664dba96c9f7692e5285a5', 'request_id': 'req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.438092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e5c871bb5664dba96c9f7692e5285a5', 'request_id': 'req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.464024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59 admin admin] The token cache is being invalidate because project 9e5c871bb5664dba96c9f7692e5285a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:15.464934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.467557 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b2c97a88-63a1-4b8c-8c1c-9c77915d4e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:15.476906 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 871/1733] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:15 2022] DELETE /identity/v3/projects/9e5c871bb5664dba96c9f7692e5285a5 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:15.479707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0b062-a46d-451a-9499-9fccfb752478 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.480164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0b062-a46d-451a-9499-9fccfb752478 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.480932 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0b062-a46d-451a-9499-9fccfb752478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.511588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:15.514285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 863/1734] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:15.517495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5f257-e0e0-4ed5-8eec-4dc7ee3c2291 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:15.562362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-fec0b062-a46d-451a-9499-9fccfb752478 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:15.564798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fec0b062-a46d-451a-9499-9fccfb752478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:15.565112 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5f257-e0e0-4ed5-8eec-4dc7ee3c2291 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:15.581730 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6eb66-e942-4f71-8b95-6f8ec35c1b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:15.583257 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6eb66-e942-4f71-8b95-6f8ec35c1b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.584289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6eb66-e942-4f71-8b95-6f8ec35c1b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.585060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6eb66-e942-4f71-8b95-6f8ec35c1b60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.587857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 872/1735] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:15.603726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ee736-1e3e-4a3f-8861-8525e9659b6b None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.604368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ee736-1e3e-4a3f-8861-8525e9659b6b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.605006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ee736-1e3e-4a3f-8861-8525e9659b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.608659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6eb66-e942-4f71-8b95-6f8ec35c1b60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:15.610039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6eb66-e942-4f71-8b95-6f8ec35c1b60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.623365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6eb66-e942-4f71-8b95-6f8ec35c1b60 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.662530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:15.680981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 864/1736] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:15.686854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6eb66-e942-4f71-8b95-6f8ec35c1b60 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:15.713726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ac7ee736-1e3e-4a3f-8861-8525e9659b6b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:15.717706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ac7ee736-1e3e-4a3f-8861-8525e9659b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:15.744831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 873/1737] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:15.745224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6eb66-e942-4f71-8b95-6f8ec35c1b60 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:15.750688 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ee736-1e3e-4a3f-8861-8525e9659b6b None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:15.771082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47322230-a7f5-45e7-9757-d2b43115afde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:15.773550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47322230-a7f5-45e7-9757-d2b43115afde admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.776079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47322230-a7f5-45e7-9757-d2b43115afde admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.776372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ee736-1e3e-4a3f-8861-8525e9659b6b None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:15.781385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47322230-a7f5-45e7-9757-d2b43115afde admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.783010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47322230-a7f5-45e7-9757-d2b43115afde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.796231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47322230-a7f5-45e7-9757-d2b43115afde admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.799377 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c7340-4269-4eb7-bcf4-3ce8eb22139f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:15.802763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c7340-4269-4eb7-bcf4-3ce8eb22139f admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.806264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c7340-4269-4eb7-bcf4-3ce8eb22139f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.806767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c7340-4269-4eb7-bcf4-3ce8eb22139f admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.811044 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c7340-4269-4eb7-bcf4-3ce8eb22139f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:15.811646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c7340-4269-4eb7-bcf4-3ce8eb22139f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.814735 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c7340-4269-4eb7-bcf4-3ce8eb22139f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.858682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 874/1738] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:15.877465 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c7340-4269-4eb7-bcf4-3ce8eb22139f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:15.929708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c7340-4269-4eb7-bcf4-3ce8eb22139f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:15.944977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e602f-ef89-4296-b513-1d8e372f86ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:15.946245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e602f-ef89-4296-b513-1d8e372f86ec admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.947358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e602f-ef89-4296-b513-1d8e372f86ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.947909 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e602f-ef89-4296-b513-1d8e372f86ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.955520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e602f-ef89-4296-b513-1d8e372f86ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.955793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 865/1739] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:15 2022] POST /identity/v3/projects => generated 371 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:15.957005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e602f-ef89-4296-b513-1d8e372f86ec admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.962092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914ac84-18fa-4743-a817-9b84c46cb4fe None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.964194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914ac84-18fa-4743-a817-9b84c46cb4fe None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.966423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914ac84-18fa-4743-a817-9b84c46cb4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.027225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0914ac84-18fa-4743-a817-9b84c46cb4fe None None] MFA Rules not processed for user `8a120d9e7e204dc5b972fd9402897b41`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:16.030106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0914ac84-18fa-4743-a817-9b84c46cb4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:16.180136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 875/1740] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:15 2022] POST /identity/v3/projects => generated 365 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:16.181037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e602f-ef89-4296-b513-1d8e372f86ec admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.219516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 866/1741] 10.222.0.40 () {58 vars in 1015 bytes} [Thu Jul 21 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:16.222867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914ac84-18fa-4743-a817-9b84c46cb4fe None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.344100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e602f-ef89-4296-b513-1d8e372f86ec admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.350705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411a826-68e3-4e2c-8f89-8a90c38effe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.351504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411a826-68e3-4e2c-8f89-8a90c38effe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.351805 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411a826-68e3-4e2c-8f89-8a90c38effe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.352089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411a826-68e3-4e2c-8f89-8a90c38effe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.352857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411a826-68e3-4e2c-8f89-8a90c38effe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.355023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411a826-68e3-4e2c-8f89-8a90c38effe2 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.395713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 876/1742] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:16 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:16.413384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411a826-68e3-4e2c-8f89-8a90c38effe2 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.472908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411a826-68e3-4e2c-8f89-8a90c38effe2 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.490627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07079d8-973e-4520-9928-6edfbce06ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.492146 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07079d8-973e-4520-9928-6edfbce06ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.492796 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07079d8-973e-4520-9928-6edfbce06ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.493271 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07079d8-973e-4520-9928-6edfbce06ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.494624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07079d8-973e-4520-9928-6edfbce06ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.497758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07079d8-973e-4520-9928-6edfbce06ed5 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.504046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914ac84-18fa-4743-a817-9b84c46cb4fe None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.516039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61cd08-f455-472a-b432-9a49c82a13f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.518117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61cd08-f455-472a-b432-9a49c82a13f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.519165 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61cd08-f455-472a-b432-9a49c82a13f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.519985 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61cd08-f455-472a-b432-9a49c82a13f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.523176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da61cd08-f455-472a-b432-9a49c82a13f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.529453 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da61cd08-f455-472a-b432-9a49c82a13f4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.547165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 877/1743] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:16.549545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07079d8-973e-4520-9928-6edfbce06ed5 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.585845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07079d8-973e-4520-9928-6edfbce06ed5 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.602784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 867/1744] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:16 2022] POST /identity/v3/users => generated 283 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:16.604104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61cd08-f455-472a-b432-9a49c82a13f4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.608858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30851a25-d13b-41a5-9661-12a92a2b5534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.610092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30851a25-d13b-41a5-9661-12a92a2b5534 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.610617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30851a25-d13b-41a5-9661-12a92a2b5534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.611139 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30851a25-d13b-41a5-9661-12a92a2b5534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.612891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30851a25-d13b-41a5-9661-12a92a2b5534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.616214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30851a25-d13b-41a5-9661-12a92a2b5534 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.634931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61cd08-f455-472a-b432-9a49c82a13f4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.641604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 878/1745] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:16.648450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb5a84-0171-41bd-9523-c2efb44d249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.648711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30851a25-d13b-41a5-9661-12a92a2b5534 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.649865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb5a84-0171-41bd-9523-c2efb44d249f admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.650343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb5a84-0171-41bd-9523-c2efb44d249f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.650809 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb5a84-0171-41bd-9523-c2efb44d249f admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.651771 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb5a84-0171-41bd-9523-c2efb44d249f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.656138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb5a84-0171-41bd-9523-c2efb44d249f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.674679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30851a25-d13b-41a5-9661-12a92a2b5534 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.675240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 868/1746] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:16.676318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb5a84-0171-41bd-9523-c2efb44d249f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.681847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2bcb5a84-0171-41bd-9523-c2efb44d249f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:16.693835 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57ad91-a4b0-48a2-b3a1-50e863388d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.696214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57ad91-a4b0-48a2-b3a1-50e863388d20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.696935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57ad91-a4b0-48a2-b3a1-50e863388d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.697644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee57ad91-a4b0-48a2-b3a1-50e863388d20 admin admin] PATH_INFO: `/v3/projects/45d539964a964635b8ebe7afb291f1d1/users/27e58ecac94c4133817d957f77355445/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.706830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb5a84-0171-41bd-9523-c2efb44d249f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.723527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf1fc8-3699-4fed-8f13-34309b209380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.725283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf1fc8-3699-4fed-8f13-34309b209380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.725634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf1fc8-3699-4fed-8f13-34309b209380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.725922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf1fc8-3699-4fed-8f13-34309b209380 admin admin] PATH_INFO: `/v3/users/d315d995684c4d7ca504a1ea7d55d61d` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.728524 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf1fc8-3699-4fed-8f13-34309b209380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d315d995684c4d7ca504a1ea7d55d61d)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.732419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf1fc8-3699-4fed-8f13-34309b209380 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.736020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57ad91-a4b0-48a2-b3a1-50e863388d20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d539964a964635b8ebe7afb291f1d1, user_id=27e58ecac94c4133817d957f77355445, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.742021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee57ad91-a4b0-48a2-b3a1-50e863388d20 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.768531 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 879/1747] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:16 2022] PUT /identity/v3/projects/45d539964a964635b8ebe7afb291f1d1/users/27e58ecac94c4133817d957f77355445/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:16.771589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57ad91-a4b0-48a2-b3a1-50e863388d20 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:16.858741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b8bf1fc8-3699-4fed-8f13-34309b209380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd315d995684c4d7ca504a1ea7d55d61d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:16.893021 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b8bf1fc8-3699-4fed-8f13-34309b209380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd315d995684c4d7ca504a1ea7d55d61d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:16.904529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:16.907340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee57ad91-a4b0-48a2-b3a1-50e863388d20 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:16.916084 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790aa132-1c9d-43e2-8cb6-73bc15e60baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:16.917443 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790aa132-1c9d-43e2-8cb6-73bc15e60baa admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.917943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790aa132-1c9d-43e2-8cb6-73bc15e60baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.918585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790aa132-1c9d-43e2-8cb6-73bc15e60baa admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.920262 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790aa132-1c9d-43e2-8cb6-73bc15e60baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:16.923022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790aa132-1c9d-43e2-8cb6-73bc15e60baa admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:16.925092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:16.931129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b8bf1fc8-3699-4fed-8f13-34309b209380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd315d995684c4d7ca504a1ea7d55d61d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:16.949788 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b8bf1fc8-3699-4fed-8f13-34309b209380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd315d995684c4d7ca504a1ea7d55d61d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:16.950779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b8bf1fc8-3699-4fed-8f13-34309b209380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd315d995684c4d7ca504a1ea7d55d61d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:16.959476 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 869/1748] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:16 2022] DELETE /identity/v3/users/d315d995684c4d7ca504a1ea7d55d61d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:16.960663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd483c-63e5-4b87-b660-e09d26aadd7d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:16.961200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd483c-63e5-4b87-b660-e09d26aadd7d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:16.961318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd483c-63e5-4b87-b660-e09d26aadd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:16.988614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 880/1749] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:16.990255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790aa132-1c9d-43e2-8cb6-73bc15e60baa admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:17.004798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-790aa132-1c9d-43e2-8cb6-73bc15e60baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:17.005529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c8fd483c-63e5-4b87-b660-e09d26aadd7d None None] MFA Rules not processed for user `27e58ecac94c4133817d957f77355445`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:17.010473 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c8fd483c-63e5-4b87-b660-e09d26aadd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:17.048802 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790aa132-1c9d-43e2-8cb6-73bc15e60baa admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:17.311381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 870/1750] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:17.321375 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd483c-63e5-4b87-b660-e09d26aadd7d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:17.334544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c8fd483c-63e5-4b87-b660-e09d26aadd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:17.394171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38655d51-8171-43c1-aad0-ff5f28a32aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:17.395824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38655d51-8171-43c1-aad0-ff5f28a32aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.399479 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38655d51-8171-43c1-aad0-ff5f28a32aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.399793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38655d51-8171-43c1-aad0-ff5f28a32aa7 admin admin] PATH_INFO: `/v3/projects/7449f6aa377b4633aaa27ad3dbc09336/users/e320bea4f4794ce5a07476ea472e4918/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.458706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38655d51-8171-43c1-aad0-ff5f28a32aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7449f6aa377b4633aaa27ad3dbc09336, user_id=e320bea4f4794ce5a07476ea472e4918, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:17.463090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38655d51-8171-43c1-aad0-ff5f28a32aa7 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:17.495954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 881/1751] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:16 2022] PUT /identity/v3/projects/7449f6aa377b4633aaa27ad3dbc09336/users/e320bea4f4794ce5a07476ea472e4918/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:17.499132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38655d51-8171-43c1-aad0-ff5f28a32aa7 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:17.511014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-38655d51-8171-43c1-aad0-ff5f28a32aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:17.540445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd483c-63e5-4b87-b660-e09d26aadd7d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:17.651951 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8835933-0edc-4d6a-b520-bd428ebf2ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:17.652868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8835933-0edc-4d6a-b520-bd428ebf2ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.662433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8835933-0edc-4d6a-b520-bd428ebf2ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.664895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8835933-0edc-4d6a-b520-bd428ebf2ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.665819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8835933-0edc-4d6a-b520-bd428ebf2ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:17.668309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8835933-0edc-4d6a-b520-bd428ebf2ea1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:17.708208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:17.723251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38655d51-8171-43c1-aad0-ff5f28a32aa7 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:17.734413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 871/1752] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:17 2022] POST /identity/v3/projects => generated 373 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:17.737195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4abecf-b356-4c15-8803-885a170faf26 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.737975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4abecf-b356-4c15-8803-885a170faf26 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.738618 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4abecf-b356-4c15-8803-885a170faf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.838141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-bb4abecf-b356-4c15-8803-885a170faf26 None None] MFA Rules not processed for user `e320bea4f4794ce5a07476ea472e4918`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:17.840139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bb4abecf-b356-4c15-8803-885a170faf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:17.957303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:17.959150 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.959893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.960614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] PATH_INFO: `/v3/projects/7923f53751164d94849aefc2ae7744b0` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.963147 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7923f53751164d94849aefc2ae7744b0)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:17.972355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:17.977224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 872/1753] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:17.978067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4abecf-b356-4c15-8803-885a170faf26 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:17.990210 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4abecf-b356-4c15-8803-885a170faf26 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:17.995333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16200075-9d77-4535-bfdd-8efb23976d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:17.996041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16200075-9d77-4535-bfdd-8efb23976d54 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.996335 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16200075-9d77-4535-bfdd-8efb23976d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.996664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16200075-9d77-4535-bfdd-8efb23976d54 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.998432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16200075-9d77-4535-bfdd-8efb23976d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.000715 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16200075-9d77-4535-bfdd-8efb23976d54 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.035899 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 873/1754] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:17 2022] POST /identity/v3/users => generated 293 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:18.037083 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16200075-9d77-4535-bfdd-8efb23976d54 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.052923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16200075-9d77-4535-bfdd-8efb23976d54 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.060313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022572f7-0a29-48df-b7ba-9e00937c33f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.061075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022572f7-0a29-48df-b7ba-9e00937c33f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.067855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022572f7-0a29-48df-b7ba-9e00937c33f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.068151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022572f7-0a29-48df-b7ba-9e00937c33f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.069017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022572f7-0a29-48df-b7ba-9e00937c33f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.072064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022572f7-0a29-48df-b7ba-9e00937c33f8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.095940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:18.144326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 874/1755] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:18 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:18.150759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022572f7-0a29-48df-b7ba-9e00937c33f8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.157506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7923f53751164d94849aefc2ae7744b0', 'request_id': 'req-d22e5393-de68-4659-98ed-862df47501d4'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.183620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7923f53751164d94849aefc2ae7744b0', 'request_id': 'req-d22e5393-de68-4659-98ed-862df47501d4'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.209610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7923f53751164d94849aefc2ae7744b0', 'request_id': 'req-d22e5393-de68-4659-98ed-862df47501d4'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.233677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] The token cache is being invalidate because project 7923f53751164d94849aefc2ae7744b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:18.234192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.236213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.240247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 882/1756] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:17 2022] DELETE /identity/v3/projects/7923f53751164d94849aefc2ae7744b0 => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:18.247409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.264883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:18.329147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022572f7-0a29-48df-b7ba-9e00937c33f8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.339109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48267365-ff83-417c-a52c-60bce86a55aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.340362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267365-ff83-417c-a52c-60bce86a55aa admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.342343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267365-ff83-417c-a52c-60bce86a55aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.342878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267365-ff83-417c-a52c-60bce86a55aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.345095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48267365-ff83-417c-a52c-60bce86a55aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.348374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48267365-ff83-417c-a52c-60bce86a55aa admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.377369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 875/1757] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:18.377369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48267365-ff83-417c-a52c-60bce86a55aa admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.383238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-48267365-ff83-417c-a52c-60bce86a55aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:18.414854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48267365-ff83-417c-a52c-60bce86a55aa admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.423799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.425609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.426173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.426726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] PATH_INFO: `/v3/users/ef25766307494bee9f326aa44d074c48` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.435412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef25766307494bee9f326aa44d074c48)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.451169 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.598422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e5393-de68-4659-98ed-862df47501d4 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.602578 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:18.614138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef25766307494bee9f326aa44d074c48'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.614471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cf7cb-036a-4419-a998-681154a1e164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.615633 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cf7cb-036a-4419-a998-681154a1e164 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.616083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cf7cb-036a-4419-a998-681154a1e164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.616496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cf7cb-036a-4419-a998-681154a1e164 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.622639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cf7cb-036a-4419-a998-681154a1e164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.626868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453cf7cb-036a-4419-a998-681154a1e164 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.633171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef25766307494bee9f326aa44d074c48'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.652899 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef25766307494bee9f326aa44d074c48'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.672082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef25766307494bee9f326aa44d074c48'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.681098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef25766307494bee9f326aa44d074c48'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:18.681384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 883/1758] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:18 2022] POST /identity/v3/users => generated 289 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:18.685076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cf7cb-036a-4419-a998-681154a1e164 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.690408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-453cf7cb-036a-4419-a998-681154a1e164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:18.696737 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 876/1759] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:18 2022] DELETE /identity/v3/users/ef25766307494bee9f326aa44d074c48 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:18.697742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.716909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:18.747190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453cf7cb-036a-4419-a998-681154a1e164 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.871677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc3382-5808-4ea2-a192-6bde593b8d00 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.885027 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911a869-c6af-4450-8a57-22aed71b35bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.885735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911a869-c6af-4450-8a57-22aed71b35bf admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.886024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911a869-c6af-4450-8a57-22aed71b35bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.886304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911a869-c6af-4450-8a57-22aed71b35bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.887057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911a869-c6af-4450-8a57-22aed71b35bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.888805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911a869-c6af-4450-8a57-22aed71b35bf admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.899753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5603d2b-997f-411f-a6c7-6d4a0766b386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.900779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5603d2b-997f-411f-a6c7-6d4a0766b386 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.901116 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5603d2b-997f-411f-a6c7-6d4a0766b386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.901341 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5603d2b-997f-411f-a6c7-6d4a0766b386 admin admin] PATH_INFO: `/v3/projects/28cf8b7948544bfc9e9a690c69da8937/users/2fa864c0a75046c1b9096584413e56b0/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.917603 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 877/1760] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:18.921317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911a869-c6af-4450-8a57-22aed71b35bf admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.938783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911a869-c6af-4450-8a57-22aed71b35bf admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.941748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5603d2b-997f-411f-a6c7-6d4a0766b386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28cf8b7948544bfc9e9a690c69da8937, user_id=2fa864c0a75046c1b9096584413e56b0, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.944587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5603d2b-997f-411f-a6c7-6d4a0766b386 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.951465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e64843-5b26-4c97-b5fd-54f0f9bd1ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.952434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e64843-5b26-4c97-b5fd-54f0f9bd1ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.952926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e64843-5b26-4c97-b5fd-54f0f9bd1ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.953311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e64843-5b26-4c97-b5fd-54f0f9bd1ee2 admin admin] PATH_INFO: `/v3/projects/c5d9274e62c64ffd9939acdd9b7546bd/users/b2910baa2e6748efac91d52a80440eba/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.965371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 884/1761] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:18 2022] PUT /identity/v3/projects/28cf8b7948544bfc9e9a690c69da8937/users/2fa864c0a75046c1b9096584413e56b0/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:18.972393 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5603d2b-997f-411f-a6c7-6d4a0766b386 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.989566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e64843-5b26-4c97-b5fd-54f0f9bd1ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d9274e62c64ffd9939acdd9b7546bd, user_id=b2910baa2e6748efac91d52a80440eba, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.994139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e64843-5b26-4c97-b5fd-54f0f9bd1ee2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.026262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 878/1762] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:18 2022] PUT /identity/v3/projects/c5d9274e62c64ffd9939acdd9b7546bd/users/b2910baa2e6748efac91d52a80440eba/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:19.031251 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e64843-5b26-4c97-b5fd-54f0f9bd1ee2 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:19.035161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:19.124156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5603d2b-997f-411f-a6c7-6d4a0766b386 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:19.209873 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e64843-5b26-4c97-b5fd-54f0f9bd1ee2 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:19.220047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2aeac-d435-40ec-bd63-02dafc9919ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:19.221345 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2aeac-d435-40ec-bd63-02dafc9919ee admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.221898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2aeac-d435-40ec-bd63-02dafc9919ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.222391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2aeac-d435-40ec-bd63-02dafc9919ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.223759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2aeac-d435-40ec-bd63-02dafc9919ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:19.228365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2aeac-d435-40ec-bd63-02dafc9919ee admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.248721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 879/1763] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:19.251741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2aeac-d435-40ec-bd63-02dafc9919ee admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:19.253255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-33a2aeac-d435-40ec-bd63-02dafc9919ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:19.263026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc92b9-708f-4663-abef-95fb28b4e387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:19.263834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc92b9-708f-4663-abef-95fb28b4e387 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.264109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc92b9-708f-4663-abef-95fb28b4e387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.264458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc92b9-708f-4663-abef-95fb28b4e387 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.269693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc92b9-708f-4663-abef-95fb28b4e387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:19.271529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc92b9-708f-4663-abef-95fb28b4e387 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.317292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 885/1764] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:19.318460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc92b9-708f-4663-abef-95fb28b4e387 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:19.319637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2aeac-d435-40ec-bd63-02dafc9919ee admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:19.336090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc92b9-708f-4663-abef-95fb28b4e387 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:19.344783 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef508f5-7027-476b-be00-98ca2c285eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:19.345862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef508f5-7027-476b-be00-98ca2c285eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.346434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef508f5-7027-476b-be00-98ca2c285eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.346732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef508f5-7027-476b-be00-98ca2c285eab admin admin] PATH_INFO: `/v3/projects/c5d9274e62c64ffd9939acdd9b7546bd/users/b2910baa2e6748efac91d52a80440eba/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.347648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554f8c6-ea0c-4cf8-ad13-c181006028e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:19.348860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554f8c6-ea0c-4cf8-ad13-c181006028e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.349065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554f8c6-ea0c-4cf8-ad13-c181006028e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.349420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554f8c6-ea0c-4cf8-ad13-c181006028e7 admin admin] PATH_INFO: `/v3/projects/90eda2a6e3fc44ef952e2bdd61f0b27d` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.349978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef508f5-7027-476b-be00-98ca2c285eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d9274e62c64ffd9939acdd9b7546bd, user_id=b2910baa2e6748efac91d52a80440eba, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:19.350946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554f8c6-ea0c-4cf8-ad13-c181006028e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90eda2a6e3fc44ef952e2bdd61f0b27d)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:19.353114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554f8c6-ea0c-4cf8-ad13-c181006028e7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.353531 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef508f5-7027-476b-be00-98ca2c285eab admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.369435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 886/1765] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:19 2022] PUT /identity/v3/projects/c5d9274e62c64ffd9939acdd9b7546bd/users/b2910baa2e6748efac91d52a80440eba/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:19.370365 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef508f5-7027-476b-be00-98ca2c285eab admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:19.419098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:19.500711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4554f8c6-ea0c-4cf8-ad13-c181006028e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90eda2a6e3fc44ef952e2bdd61f0b27d', 'request_id': 'req-4554f8c6-ea0c-4cf8-ad13-c181006028e7'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:19.513798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4554f8c6-ea0c-4cf8-ad13-c181006028e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90eda2a6e3fc44ef952e2bdd61f0b27d', 'request_id': 'req-4554f8c6-ea0c-4cf8-ad13-c181006028e7'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:19.528935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4554f8c6-ea0c-4cf8-ad13-c181006028e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90eda2a6e3fc44ef952e2bdd61f0b27d', 'request_id': 'req-4554f8c6-ea0c-4cf8-ad13-c181006028e7'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:19.541073 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef508f5-7027-476b-be00-98ca2c285eab admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:19.556985 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4554f8c6-ea0c-4cf8-ad13-c181006028e7 admin admin] The token cache is being invalidate because project 90eda2a6e3fc44ef952e2bdd61f0b27d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:19.557288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4554f8c6-ea0c-4cf8-ad13-c181006028e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:19.560118 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4554f8c6-ea0c-4cf8-ad13-c181006028e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:19.560384 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 880/1766] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:19 2022] DELETE /identity/v3/projects/90eda2a6e3fc44ef952e2bdd61f0b27d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:19.564564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de01f34-fa77-4a1d-96e3-a3628ca24321 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.564952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de01f34-fa77-4a1d-96e3-a3628ca24321 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.575257 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de01f34-fa77-4a1d-96e3-a3628ca24321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.629588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2de01f34-fa77-4a1d-96e3-a3628ca24321 None None] MFA Rules not processed for user `b2910baa2e6748efac91d52a80440eba`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:19.632101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2de01f34-fa77-4a1d-96e3-a3628ca24321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:19.670548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73b162-77a8-4a55-8e41-d20ba179d147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:19.671661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73b162-77a8-4a55-8e41-d20ba179d147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.672047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73b162-77a8-4a55-8e41-d20ba179d147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.672424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73b162-77a8-4a55-8e41-d20ba179d147 admin admin] PATH_INFO: `/v3/projects/28cf8b7948544bfc9e9a690c69da8937/users/2fa864c0a75046c1b9096584413e56b0/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.676698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73b162-77a8-4a55-8e41-d20ba179d147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28cf8b7948544bfc9e9a690c69da8937, user_id=2fa864c0a75046c1b9096584413e56b0, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:19.687173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73b162-77a8-4a55-8e41-d20ba179d147 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:19.703951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:19.724597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 887/1767] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:19 2022] PUT /identity/v3/projects/28cf8b7948544bfc9e9a690c69da8937/users/2fa864c0a75046c1b9096584413e56b0/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:19.724597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedbf8c-cdd2-45b1-8037-bcc847de5a8f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.724597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedbf8c-cdd2-45b1-8037-bcc847de5a8f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.724597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedbf8c-cdd2-45b1-8037-bcc847de5a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.808267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-5aedbf8c-cdd2-45b1-8037-bcc847de5a8f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:19.812382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5aedbf8c-cdd2-45b1-8037-bcc847de5a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:19.854916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 881/1768] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:19.854916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de4f9f-fc8a-4b1f-bd1e-9099e9761bb8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.854916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de4f9f-fc8a-4b1f-bd1e-9099e9761bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.854916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de4f9f-fc8a-4b1f-bd1e-9099e9761bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.950735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b0de4f9f-fc8a-4b1f-bd1e-9099e9761bb8 None None] MFA Rules not processed for user `2fa864c0a75046c1b9096584413e56b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:19.953857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b0de4f9f-fc8a-4b1f-bd1e-9099e9761bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:20.035009 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 888/1769] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:20.038023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedbf8c-cdd2-45b1-8037-bcc847de5a8f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.053216 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5aedbf8c-cdd2-45b1-8037-bcc847de5a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:20.180846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aedbf8c-cdd2-45b1-8037-bcc847de5a8f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:20.188404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278c65e-861d-45ba-a500-5687b1b653b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:20.190840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278c65e-861d-45ba-a500-5687b1b653b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.191751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278c65e-861d-45ba-a500-5687b1b653b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.192273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278c65e-861d-45ba-a500-5687b1b653b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.195777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c278c65e-861d-45ba-a500-5687b1b653b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:20.225197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278c65e-861d-45ba-a500-5687b1b653b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:20.230476 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278c65e-861d-45ba-a500-5687b1b653b7 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:20.282905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 889/1770] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:20.285007 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278c65e-861d-45ba-a500-5687b1b653b7 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.292204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c278c65e-861d-45ba-a500-5687b1b653b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:20.320695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 882/1771] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:20.323080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de4f9f-fc8a-4b1f-bd1e-9099e9761bb8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.327074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b0de4f9f-fc8a-4b1f-bd1e-9099e9761bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:20.359587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de4f9f-fc8a-4b1f-bd1e-9099e9761bb8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:20.376027 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43465fd-ae1b-4ca6-8033-cc37801cf02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:20.377137 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43465fd-ae1b-4ca6-8033-cc37801cf02d service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.377448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43465fd-ae1b-4ca6-8033-cc37801cf02d service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.377965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43465fd-ae1b-4ca6-8033-cc37801cf02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.405042 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43465fd-ae1b-4ca6-8033-cc37801cf02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:20.408648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43465fd-ae1b-4ca6-8033-cc37801cf02d service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:20.434850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 883/1772] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:20.439330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddfb86-2c3d-49ec-b1d6-bd4cac25d61c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.439640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddfb86-2c3d-49ec-b1d6-bd4cac25d61c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.439919 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddfb86-2c3d-49ec-b1d6-bd4cac25d61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.492705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:20.517348 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7cddfb86-2c3d-49ec-b1d6-bd4cac25d61c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:20.518583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7cddfb86-2c3d-49ec-b1d6-bd4cac25d61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:20.533249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 884/1773] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:20.540775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278c65e-861d-45ba-a500-5687b1b653b7 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:20.545813 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddfb86-2c3d-49ec-b1d6-bd4cac25d61c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.560027 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1daf073-6ef0-4415-acd1-5edaca8f1e15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:20.561277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1daf073-6ef0-4415-acd1-5edaca8f1e15 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.561682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1daf073-6ef0-4415-acd1-5edaca8f1e15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.565091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddfb86-2c3d-49ec-b1d6-bd4cac25d61c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:20.565625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1daf073-6ef0-4415-acd1-5edaca8f1e15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.585021 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b856eda-6e42-4f59-9287-43cbb1bf25c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:20.585891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b856eda-6e42-4f59-9287-43cbb1bf25c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.586248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b856eda-6e42-4f59-9287-43cbb1bf25c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.586532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b856eda-6e42-4f59-9287-43cbb1bf25c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.588108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b856eda-6e42-4f59-9287-43cbb1bf25c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:20.589619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b856eda-6e42-4f59-9287-43cbb1bf25c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:20.593172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b856eda-6e42-4f59-9287-43cbb1bf25c5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:20.650668 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 885/1774] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:20.655008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b856eda-6e42-4f59-9287-43cbb1bf25c5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.661014 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1b856eda-6e42-4f59-9287-43cbb1bf25c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:20.695587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b856eda-6e42-4f59-9287-43cbb1bf25c5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:20.701168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300e8c0-48f0-471c-a48a-120f7790aa35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:20.702224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300e8c0-48f0-471c-a48a-120f7790aa35 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.702557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300e8c0-48f0-471c-a48a-120f7790aa35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.702854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300e8c0-48f0-471c-a48a-120f7790aa35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.725328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300e8c0-48f0-471c-a48a-120f7790aa35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:20.727299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300e8c0-48f0-471c-a48a-120f7790aa35 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:20.748037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 886/1775] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:20.749031 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300e8c0-48f0-471c-a48a-120f7790aa35 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.761613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300e8c0-48f0-471c-a48a-120f7790aa35 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:20.767970 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab82da0-49fc-4f46-aef4-1ab14d7f85e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:20.768784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab82da0-49fc-4f46-aef4-1ab14d7f85e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.769072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab82da0-49fc-4f46-aef4-1ab14d7f85e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.769351 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab82da0-49fc-4f46-aef4-1ab14d7f85e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.770075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab82da0-49fc-4f46-aef4-1ab14d7f85e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:20.772111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab82da0-49fc-4f46-aef4-1ab14d7f85e7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:20.774881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:20.785284 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1daf073-6ef0-4415-acd1-5edaca8f1e15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:20.787600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1daf073-6ef0-4415-acd1-5edaca8f1e15 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:20.845280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 887/1776] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:20 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:20.854192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 890/1777] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:20.854493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab82da0-49fc-4f46-aef4-1ab14d7f85e7 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.985714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab82da0-49fc-4f46-aef4-1ab14d7f85e7 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:21.009900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48827a-4108-45da-a45e-00525604ae18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:21.010697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48827a-4108-45da-a45e-00525604ae18 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.011230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48827a-4108-45da-a45e-00525604ae18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.011497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48827a-4108-45da-a45e-00525604ae18 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.013388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48827a-4108-45da-a45e-00525604ae18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:21.015488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48827a-4108-45da-a45e-00525604ae18 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:21.043209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1daf073-6ef0-4415-acd1-5edaca8f1e15 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:21.057441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1daf073-6ef0-4415-acd1-5edaca8f1e15 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:21.066653 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 888/1778] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:20 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:21.071704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48827a-4108-45da-a45e-00525604ae18 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:21.089615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48827a-4108-45da-a45e-00525604ae18 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:21.098935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35270373-c173-4d69-a200-444f6e45f449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:21.099955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35270373-c173-4d69-a200-444f6e45f449 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.100725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35270373-c173-4d69-a200-444f6e45f449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.101168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35270373-c173-4d69-a200-444f6e45f449 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.102262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35270373-c173-4d69-a200-444f6e45f449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:21.104674 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35270373-c173-4d69-a200-444f6e45f449 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:21.123928 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 889/1779] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:21.129854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35270373-c173-4d69-a200-444f6e45f449 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:21.143346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35270373-c173-4d69-a200-444f6e45f449 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:21.149396 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa61a85-3d0f-4600-9090-ca5a2fac1900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:21.150388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa61a85-3d0f-4600-9090-ca5a2fac1900 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.150894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa61a85-3d0f-4600-9090-ca5a2fac1900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.151289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa61a85-3d0f-4600-9090-ca5a2fac1900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.152226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa61a85-3d0f-4600-9090-ca5a2fac1900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:21.154844 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa61a85-3d0f-4600-9090-ca5a2fac1900 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:21.171887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138fd94-0f02-4cc6-a70c-dc4604087bf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:21.172691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138fd94-0f02-4cc6-a70c-dc4604087bf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.173068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138fd94-0f02-4cc6-a70c-dc4604087bf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.173356 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138fd94-0f02-4cc6-a70c-dc4604087bf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.174250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 890/1780] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:21.178898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa61a85-3d0f-4600-9090-ca5a2fac1900 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:21.191327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c138fd94-0f02-4cc6-a70c-dc4604087bf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:21.205993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa61a85-3d0f-4600-9090-ca5a2fac1900 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:21.213657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4416d8-b55d-489d-a89c-7ad673c70da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:21.214772 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4416d8-b55d-489d-a89c-7ad673c70da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.215134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4416d8-b55d-489d-a89c-7ad673c70da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.216937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4416d8-b55d-489d-a89c-7ad673c70da9 admin admin] PATH_INFO: `/v3/projects/5293d747e36748fb975f672998c982de/users/72f4e9ebd65b44f7b9978a9fdc246c10/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.240949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4416d8-b55d-489d-a89c-7ad673c70da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5293d747e36748fb975f672998c982de, user_id=72f4e9ebd65b44f7b9978a9fdc246c10, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:21.245456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4416d8-b55d-489d-a89c-7ad673c70da9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:21.266330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 891/1781] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:21 2022] PUT /identity/v3/projects/5293d747e36748fb975f672998c982de/users/72f4e9ebd65b44f7b9978a9fdc246c10/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:21.281538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653f127-e42b-436d-9831-6af719db675e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.282267 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653f127-e42b-436d-9831-6af719db675e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.282866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653f127-e42b-436d-9831-6af719db675e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.334646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138fd94-0f02-4cc6-a70c-dc4604087bf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:21.337039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138fd94-0f02-4cc6-a70c-dc4604087bf2 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:21.346605 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b653f127-e42b-436d-9831-6af719db675e None None] MFA Rules not processed for user `72f4e9ebd65b44f7b9978a9fdc246c10`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:21.347940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b653f127-e42b-436d-9831-6af719db675e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:21.362476 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 891/1782] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:21.424593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:21.465785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 892/1783] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:21.471256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138fd94-0f02-4cc6-a70c-dc4604087bf2 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:21.516832 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138fd94-0f02-4cc6-a70c-dc4604087bf2 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:21.741390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb111e3-e4c6-43f5-a9b7-7bcee058e49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:21.747365 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb111e3-e4c6-43f5-a9b7-7bcee058e49f admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:21.752425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb111e3-e4c6-43f5-a9b7-7bcee058e49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:21.758930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb111e3-e4c6-43f5-a9b7-7bcee058e49f admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:21.765956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb111e3-e4c6-43f5-a9b7-7bcee058e49f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:21.789332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb111e3-e4c6-43f5-a9b7-7bcee058e49f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:21.915921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 892/1784] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:21 2022] POST /identity/v3/projects => generated 385 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:21.920589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653f127-e42b-436d-9831-6af719db675e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:21.934595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653f127-e42b-436d-9831-6af719db675e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:22.078224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e5d62-0e0a-493b-a22b-a117215bb123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:22.078923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e5d62-0e0a-493b-a22b-a117215bb123 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:22.079216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e5d62-0e0a-493b-a22b-a117215bb123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:22.079491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e5d62-0e0a-493b-a22b-a117215bb123 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:22.080253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e5d62-0e0a-493b-a22b-a117215bb123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:22.082385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e5d62-0e0a-493b-a22b-a117215bb123 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:22.139781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 893/1785] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:21 2022] POST /identity/v3/users => generated 299 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:22.144912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb111e3-e4c6-43f5-a9b7-7bcee058e49f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:22.159543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb111e3-e4c6-43f5-a9b7-7bcee058e49f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:22.172747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03092826-67b2-44a2-bd57-78c9ace85a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:22.173455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03092826-67b2-44a2-bd57-78c9ace85a13 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:22.173914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03092826-67b2-44a2-bd57-78c9ace85a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:22.175629 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03092826-67b2-44a2-bd57-78c9ace85a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:22.177107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03092826-67b2-44a2-bd57-78c9ace85a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:22.184886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03092826-67b2-44a2-bd57-78c9ace85a13 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:22.216806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 893/1786] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:22.218724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e5d62-0e0a-493b-a22b-a117215bb123 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:22.289379 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e5d62-0e0a-493b-a22b-a117215bb123 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:22.314521 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5beba1-4d42-4567-8232-7c9a66ac3ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:22.316590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5beba1-4d42-4567-8232-7c9a66ac3ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:22.317403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5beba1-4d42-4567-8232-7c9a66ac3ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:22.317878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5beba1-4d42-4567-8232-7c9a66ac3ee9 admin admin] PATH_INFO: `/v3/projects/45642dd64390418c82536819ddbc5b2c/users/3b71cebb993a4f6b82a8145972fb3c04/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:22.357677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5beba1-4d42-4567-8232-7c9a66ac3ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45642dd64390418c82536819ddbc5b2c, user_id=3b71cebb993a4f6b82a8145972fb3c04, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:22.367304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5beba1-4d42-4567-8232-7c9a66ac3ee9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:22.440936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03092826-67b2-44a2-bd57-78c9ace85a13 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:22.444845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 894/1787] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:22 2022] PUT /identity/v3/projects/45642dd64390418c82536819ddbc5b2c/users/3b71cebb993a4f6b82a8145972fb3c04/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:22.453392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5beba1-4d42-4567-8232-7c9a66ac3ee9 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:22.455721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3e5beba1-4d42-4567-8232-7c9a66ac3ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:22.458257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03092826-67b2-44a2-bd57-78c9ace85a13 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:22.522044 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:22.645876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76276606-e0e4-4499-a5d2-e2389d71ec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:22.646740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76276606-e0e4-4499-a5d2-e2389d71ec47 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:22.646964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76276606-e0e4-4499-a5d2-e2389d71ec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:22.647321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76276606-e0e4-4499-a5d2-e2389d71ec47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:22.648170 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76276606-e0e4-4499-a5d2-e2389d71ec47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:22.674618 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76276606-e0e4-4499-a5d2-e2389d71ec47 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:22.732879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 894/1788] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:22.734144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76276606-e0e4-4499-a5d2-e2389d71ec47 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:22.779495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76276606-e0e4-4499-a5d2-e2389d71ec47 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:22.803088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca361d-47c3-4503-812c-500bfbaaebb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:22.806811 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca361d-47c3-4503-812c-500bfbaaebb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:22.807414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca361d-47c3-4503-812c-500bfbaaebb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:22.807806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca361d-47c3-4503-812c-500bfbaaebb4 admin admin] PATH_INFO: `/v3/projects/45642dd64390418c82536819ddbc5b2c/users/3b71cebb993a4f6b82a8145972fb3c04/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:22.828990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca361d-47c3-4503-812c-500bfbaaebb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45642dd64390418c82536819ddbc5b2c, user_id=3b71cebb993a4f6b82a8145972fb3c04, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:22.831860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca361d-47c3-4503-812c-500bfbaaebb4 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:22.848981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5beba1-4d42-4567-8232-7c9a66ac3ee9 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:22.881700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a36d8-bbf5-4637-9f2e-a5ee8e432480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:22.882885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a36d8-bbf5-4637-9f2e-a5ee8e432480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:22.883177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a36d8-bbf5-4637-9f2e-a5ee8e432480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:22.883454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a36d8-bbf5-4637-9f2e-a5ee8e432480 admin admin] PATH_INFO: `/v3/users/e320bea4f4794ce5a07476ea472e4918` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:22.889500 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a36d8-bbf5-4637-9f2e-a5ee8e432480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e320bea4f4794ce5a07476ea472e4918)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:22.901861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a36d8-bbf5-4637-9f2e-a5ee8e432480 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:22.915864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 895/1789] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:22 2022] PUT /identity/v3/projects/45642dd64390418c82536819ddbc5b2c/users/3b71cebb993a4f6b82a8145972fb3c04/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:22.915864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a78681-9ab4-4e62-a750-10810ea5ca6d None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:22.915864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a78681-9ab4-4e62-a750-10810ea5ca6d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:22.915864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a78681-9ab4-4e62-a750-10810ea5ca6d None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:22.916681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 896/1790] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 22:34:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:22.922473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74ac1f-5978-4a9e-97de-c03fe64c86d9 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:22.923996 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74ac1f-5978-4a9e-97de-c03fe64c86d9 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:22.924386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74ac1f-5978-4a9e-97de-c03fe64c86d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.034258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-5b74ac1f-5978-4a9e-97de-c03fe64c86d9 None None] MFA Rules not processed for user `3b71cebb993a4f6b82a8145972fb3c04`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:23.035703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5b74ac1f-5978-4a9e-97de-c03fe64c86d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:23.045117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9e5a36d8-bbf5-4637-9f2e-a5ee8e432480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e320bea4f4794ce5a07476ea472e4918'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:23.055025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:23.062857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9e5a36d8-bbf5-4637-9f2e-a5ee8e432480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e320bea4f4794ce5a07476ea472e4918'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:23.112079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9e5a36d8-bbf5-4637-9f2e-a5ee8e432480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e320bea4f4794ce5a07476ea472e4918'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:23.140762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9e5a36d8-bbf5-4637-9f2e-a5ee8e432480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e320bea4f4794ce5a07476ea472e4918'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:23.141463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9e5a36d8-bbf5-4637-9f2e-a5ee8e432480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e320bea4f4794ce5a07476ea472e4918'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:23.166788 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 895/1791] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:22 2022] DELETE /identity/v3/users/e320bea4f4794ce5a07476ea472e4918 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:23.169615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4c388-2e79-493f-a82b-36c56506ec2a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.170037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4c388-2e79-493f-a82b-36c56506ec2a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.170330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4c388-2e79-493f-a82b-36c56506ec2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.225078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f2a4c388-2e79-493f-a82b-36c56506ec2a None None] MFA Rules not processed for user `c77f6ccc7cb84c04ac532007a04a0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:23.226621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f2a4c388-2e79-493f-a82b-36c56506ec2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:23.234010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 897/1792] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:23.235576 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35e812-ae8f-452a-9120-f35a4c18bdea None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.235878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35e812-ae8f-452a-9120-f35a4c18bdea None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.236165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35e812-ae8f-452a-9120-f35a4c18bdea None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.239357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 898/1793] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 22:34:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:23.242099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162c10f-cf0d-4041-a8c4-4376f26fac17 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.242659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162c10f-cf0d-4041-a8c4-4376f26fac17 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.245121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162c10f-cf0d-4041-a8c4-4376f26fac17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.298754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-5162c10f-cf0d-4041-a8c4-4376f26fac17 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:23.300391 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5162c10f-cf0d-4041-a8c4-4376f26fac17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:23.374228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 896/1794] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 22:34:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:23.377330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4c388-2e79-493f-a82b-36c56506ec2a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:23.379787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f2a4c388-2e79-493f-a82b-36c56506ec2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:23.453965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 899/1795] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:23.456141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7295c-0474-4bd7-b7aa-46f2bb248ebc None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.456452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7295c-0474-4bd7-b7aa-46f2bb248ebc None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.456957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7295c-0474-4bd7-b7aa-46f2bb248ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.482206 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-06e7295c-0474-4bd7-b7aa-46f2bb248ebc None None] MFA Rules not processed for user `c77f6ccc7cb84c04ac532007a04a0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:23.484062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-06e7295c-0474-4bd7-b7aa-46f2bb248ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:23.500917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 900/1796] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 22:34:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:23.509374 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7295c-0474-4bd7-b7aa-46f2bb248ebc None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:23.513298 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-06e7295c-0474-4bd7-b7aa-46f2bb248ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:23.523727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4c388-2e79-493f-a82b-36c56506ec2a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:23.532651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a63e3-6fd1-4743-baf2-a508b3cd526e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:23.534164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a63e3-6fd1-4743-baf2-a508b3cd526e service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.534790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a63e3-6fd1-4743-baf2-a508b3cd526e service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.535401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a63e3-6fd1-4743-baf2-a508b3cd526e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.570588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7295c-0474-4bd7-b7aa-46f2bb248ebc None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:23.593371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3ce98-1db2-4c68-8404-f6d4258e90e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:23.596990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3ce98-1db2-4c68-8404-f6d4258e90e6 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.597590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3ce98-1db2-4c68-8404-f6d4258e90e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.598045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3ce98-1db2-4c68-8404-f6d4258e90e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.611630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3ce98-1db2-4c68-8404-f6d4258e90e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:23.613675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3ce98-1db2-4c68-8404-f6d4258e90e6 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:23.650436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 901/1797] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:23.651649 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3ce98-1db2-4c68-8404-f6d4258e90e6 service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:23.655697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5af3ce98-1db2-4c68-8404-f6d4258e90e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:23.688888 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:23.746071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a63e3-6fd1-4743-baf2-a508b3cd526e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:23.748225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a63e3-6fd1-4743-baf2-a508b3cd526e service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:23.781864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3ce98-1db2-4c68-8404-f6d4258e90e6 service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:23.787730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427ee1c-82cc-4cb1-bcad-c72e15c0d8f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:23.788403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427ee1c-82cc-4cb1-bcad-c72e15c0d8f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.788822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427ee1c-82cc-4cb1-bcad-c72e15c0d8f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.789113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427ee1c-82cc-4cb1-bcad-c72e15c0d8f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.794320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 897/1798] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:23.795533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a63e3-6fd1-4743-baf2-a508b3cd526e service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:23.797150 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7c0a63e3-6fd1-4743-baf2-a508b3cd526e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:23.807562 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427ee1c-82cc-4cb1-bcad-c72e15c0d8f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:23.809830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427ee1c-82cc-4cb1-bcad-c72e15c0d8f5 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:23.821718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 902/1799] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:23.842195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a63e3-6fd1-4743-baf2-a508b3cd526e service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:23.849919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427ee1c-82cc-4cb1-bcad-c72e15c0d8f5 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:23.851988 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe24f5e-461e-4736-8acd-bf8769d21fca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:23.852352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2427ee1c-82cc-4cb1-bcad-c72e15c0d8f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:23.853208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe24f5e-461e-4736-8acd-bf8769d21fca service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.853686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe24f5e-461e-4736-8acd-bf8769d21fca service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.854226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe24f5e-461e-4736-8acd-bf8769d21fca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.871245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe24f5e-461e-4736-8acd-bf8769d21fca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:23.874850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe24f5e-461e-4736-8acd-bf8769d21fca service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:23.883938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:23.903946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 898/1800] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:23.932117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427ee1c-82cc-4cb1-bcad-c72e15c0d8f5 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:23.952949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:23.955111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.957183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.958175 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] PATH_INFO: `/v3/users/72f4e9ebd65b44f7b9978a9fdc246c10` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.961561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f4e9ebd65b44f7b9978a9fdc246c10)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:23.964403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:24.127285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f4e9ebd65b44f7b9978a9fdc246c10'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:24.151553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f4e9ebd65b44f7b9978a9fdc246c10'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:24.153382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f4e9ebd65b44f7b9978a9fdc246c10'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:24.177698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f4e9ebd65b44f7b9978a9fdc246c10'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:24.179882 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f4e9ebd65b44f7b9978a9fdc246c10'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:24.273195 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 903/1801] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:23 2022] DELETE /identity/v3/users/72f4e9ebd65b44f7b9978a9fdc246c10 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:24.280094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc545f63-f329-47df-a14a-7895b95cddb9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.280940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc545f63-f329-47df-a14a-7895b95cddb9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.281469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc545f63-f329-47df-a14a-7895b95cddb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.325294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cc545f63-f329-47df-a14a-7895b95cddb9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:24.331745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cc545f63-f329-47df-a14a-7895b95cddb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:24.595746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 899/1802] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:24.674902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:24.676639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:24.905723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad485c2-01fc-4e67-9cbf-6393e3887366 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:24.915460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3b7d8-f89d-478c-b5e5-a7515be56dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:24.916274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3b7d8-f89d-478c-b5e5-a7515be56dfc service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.919114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3b7d8-f89d-478c-b5e5-a7515be56dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.919114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3b7d8-f89d-478c-b5e5-a7515be56dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.981705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3b7d8-f89d-478c-b5e5-a7515be56dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:24.987515 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3b7d8-f89d-478c-b5e5-a7515be56dfc service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.024801 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 904/1803] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:25.235611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc545f63-f329-47df-a14a-7895b95cddb9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.237175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cc545f63-f329-47df-a14a-7895b95cddb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:25.281337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc545f63-f329-47df-a14a-7895b95cddb9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:25.303646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:25.305188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:25.305718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:25.306179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] PATH_INFO: `/v3/projects/7449f6aa377b4633aaa27ad3dbc09336` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.308612 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7449f6aa377b4633aaa27ad3dbc09336)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.314821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.522779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7449f6aa377b4633aaa27ad3dbc09336', 'request_id': 'req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:25.535750 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7449f6aa377b4633aaa27ad3dbc09336', 'request_id': 'req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:25.578707 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7449f6aa377b4633aaa27ad3dbc09336', 'request_id': 'req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:25.629161 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] The token cache is being invalidate because project 7449f6aa377b4633aaa27ad3dbc09336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:25.630992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:25.635272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:25.638471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 900/1804] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:25 2022] DELETE /identity/v3/projects/7449f6aa377b4633aaa27ad3dbc09336 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:25.643401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3b7d8-f89d-478c-b5e5-a7515be56dfc service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.708594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3b7d8-f89d-478c-b5e5-a7515be56dfc service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:25.815948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:25.885640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919b8b2-5163-49c8-b531-133558c7e949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:25.887816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919b8b2-5163-49c8-b531-133558c7e949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:25.888337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919b8b2-5163-49c8-b531-133558c7e949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:25.889024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919b8b2-5163-49c8-b531-133558c7e949 admin admin] PATH_INFO: `/v3/users/b2910baa2e6748efac91d52a80440eba` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.896544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919b8b2-5163-49c8-b531-133558c7e949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2910baa2e6748efac91d52a80440eba)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.900450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919b8b2-5163-49c8-b531-133558c7e949 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:26.074060 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e919b8b2-5163-49c8-b531-133558c7e949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2910baa2e6748efac91d52a80440eba'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:26.097080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e919b8b2-5163-49c8-b531-133558c7e949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2910baa2e6748efac91d52a80440eba'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:26.098182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e919b8b2-5163-49c8-b531-133558c7e949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2910baa2e6748efac91d52a80440eba'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:26.129503 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e919b8b2-5163-49c8-b531-133558c7e949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2910baa2e6748efac91d52a80440eba'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:26.130400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e919b8b2-5163-49c8-b531-133558c7e949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2910baa2e6748efac91d52a80440eba'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:26.189011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 905/1805] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:25 2022] DELETE /identity/v3/users/b2910baa2e6748efac91d52a80440eba => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:26.721109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:26.724894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:26.787441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0327aa4-0bbc-4a5f-847c-da2395daa4b5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:27.069082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:27.071447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:27.071905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:27.072555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] PATH_INFO: `/v3/users/98b0cb05204c4867a73ac4d1180ed005` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:27.089425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98b0cb05204c4867a73ac4d1180ed005)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:27.098369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:27.192241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:27.281816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98b0cb05204c4867a73ac4d1180ed005'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.303215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98b0cb05204c4867a73ac4d1180ed005'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.352704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98b0cb05204c4867a73ac4d1180ed005'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.370116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98b0cb05204c4867a73ac4d1180ed005'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.372259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98b0cb05204c4867a73ac4d1180ed005'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.395820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 901/1806] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:26 2022] DELETE /identity/v3/users/98b0cb05204c4867a73ac4d1180ed005 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:27.402544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8115ea-7bff-4176-8e1d-023421b4f293 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:27.403941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8115ea-7bff-4176-8e1d-023421b4f293 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:27.403941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8115ea-7bff-4176-8e1d-023421b4f293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:27.445664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-aa8115ea-7bff-4176-8e1d-023421b4f293 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:27.448906 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-aa8115ea-7bff-4176-8e1d-023421b4f293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:27.651490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 906/1807] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:27.688178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:27.691553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:27.724080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8115ea-7bff-4176-8e1d-023421b4f293 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:27.726382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-aa8115ea-7bff-4176-8e1d-023421b4f293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:27.766336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8115ea-7bff-4176-8e1d-023421b4f293 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:27.773087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64dd24d-58af-4d1f-8fc1-860aea73359d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:27.774287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64dd24d-58af-4d1f-8fc1-860aea73359d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:27.774647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64dd24d-58af-4d1f-8fc1-860aea73359d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:27.774999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64dd24d-58af-4d1f-8fc1-860aea73359d admin admin] PATH_INFO: `/v3/projects/c5d9274e62c64ffd9939acdd9b7546bd` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:27.776641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64dd24d-58af-4d1f-8fc1-860aea73359d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5d9274e62c64ffd9939acdd9b7546bd)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:27.778986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64dd24d-58af-4d1f-8fc1-860aea73359d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:27.794563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d898382-d7b4-451f-b532-5ccb9db509e4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:27.801984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf28cce-6136-4609-af3e-7ebd671c13c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:27.802421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf28cce-6136-4609-af3e-7ebd671c13c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:27.802721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf28cce-6136-4609-af3e-7ebd671c13c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:27.803143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf28cce-6136-4609-af3e-7ebd671c13c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:27.817597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf28cce-6136-4609-af3e-7ebd671c13c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:27.819935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf28cce-6136-4609-af3e-7ebd671c13c0 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:27.833060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 902/1808] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:27.877402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:27.890425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a64dd24d-58af-4d1f-8fc1-860aea73359d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5d9274e62c64ffd9939acdd9b7546bd', 'request_id': 'req-a64dd24d-58af-4d1f-8fc1-860aea73359d'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.911461 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a64dd24d-58af-4d1f-8fc1-860aea73359d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5d9274e62c64ffd9939acdd9b7546bd', 'request_id': 'req-a64dd24d-58af-4d1f-8fc1-860aea73359d'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.924285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a64dd24d-58af-4d1f-8fc1-860aea73359d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5d9274e62c64ffd9939acdd9b7546bd', 'request_id': 'req-a64dd24d-58af-4d1f-8fc1-860aea73359d'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.954417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a64dd24d-58af-4d1f-8fc1-860aea73359d admin admin] The token cache is being invalidate because project c5d9274e62c64ffd9939acdd9b7546bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:27.955065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a64dd24d-58af-4d1f-8fc1-860aea73359d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.956533 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a64dd24d-58af-4d1f-8fc1-860aea73359d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:27.961322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 907/1809] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:27 2022] DELETE /identity/v3/projects/c5d9274e62c64ffd9939acdd9b7546bd => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:27.967727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b813a95-9e1f-48ac-a9ee-83bf4fc2621d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:27.968381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b813a95-9e1f-48ac-a9ee-83bf4fc2621d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:27.968718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b813a95-9e1f-48ac-a9ee-83bf4fc2621d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.000435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1b813a95-9e1f-48ac-a9ee-83bf4fc2621d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:28.008739 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1b813a95-9e1f-48ac-a9ee-83bf4fc2621d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:28.183645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 903/1810] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:28.188223 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a8231-6c9c-43cd-a589-f8ed13797282 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.188663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a8231-6c9c-43cd-a589-f8ed13797282 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.188837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a8231-6c9c-43cd-a589-f8ed13797282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.201781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b813a95-9e1f-48ac-a9ee-83bf4fc2621d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:28.205818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1b813a95-9e1f-48ac-a9ee-83bf4fc2621d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:28.231815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-d29a8231-6c9c-43cd-a589-f8ed13797282 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:28.233897 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b813a95-9e1f-48ac-a9ee-83bf4fc2621d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:28.235069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d29a8231-6c9c-43cd-a589-f8ed13797282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:28.253888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:28.254776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.255059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.255392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] PATH_INFO: `/v3/projects/5293d747e36748fb975f672998c982de` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.256748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5293d747e36748fb975f672998c982de)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:28.259224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:28.259609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 908/1811] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:28.267128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a8231-6c9c-43cd-a589-f8ed13797282 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:28.285751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a8231-6c9c-43cd-a589-f8ed13797282 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:28.296976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d9917-7f46-430e-b314-73be55358276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:28.298523 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d9917-7f46-430e-b314-73be55358276 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.298921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d9917-7f46-430e-b314-73be55358276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.299245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d9917-7f46-430e-b314-73be55358276 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.300507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d9917-7f46-430e-b314-73be55358276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:28.301033 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d9917-7f46-430e-b314-73be55358276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:28.304099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d9917-7f46-430e-b314-73be55358276 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:28.345238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 909/1812] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:28.347144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d9917-7f46-430e-b314-73be55358276 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:28.348858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:28.371519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d9917-7f46-430e-b314-73be55358276 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:28.373024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5293d747e36748fb975f672998c982de', 'request_id': 'req-cfe8c9e9-686f-4748-94da-77605f806743'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:28.397823 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5293d747e36748fb975f672998c982de', 'request_id': 'req-cfe8c9e9-686f-4748-94da-77605f806743'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:28.416334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5293d747e36748fb975f672998c982de', 'request_id': 'req-cfe8c9e9-686f-4748-94da-77605f806743'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:28.441114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] The token cache is being invalidate because project 5293d747e36748fb975f672998c982de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:28.441322 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:28.442146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:28.444240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 904/1813] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:28 2022] DELETE /identity/v3/projects/5293d747e36748fb975f672998c982de => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:28.449822 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:28.454711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:28.605250 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24409f12-f35e-463e-8ac0-2d2bd8b0c6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:28.607163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24409f12-f35e-463e-8ac0-2d2bd8b0c6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.607163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24409f12-f35e-463e-8ac0-2d2bd8b0c6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.607163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24409f12-f35e-463e-8ac0-2d2bd8b0c6c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.610608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24409f12-f35e-463e-8ac0-2d2bd8b0c6c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:28.617770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24409f12-f35e-463e-8ac0-2d2bd8b0c6c9 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:28.676644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 910/1814] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:28 2022] POST /identity/v3/projects => generated 353 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:28.678324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24409f12-f35e-463e-8ac0-2d2bd8b0c6c9 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:28.679288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8c9e9-686f-4748-94da-77605f806743 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:28.680835 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-24409f12-f35e-463e-8ac0-2d2bd8b0c6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:28.807970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24409f12-f35e-463e-8ac0-2d2bd8b0c6c9 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:28.817709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:28.818607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.818894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.819289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] PATH_INFO: `/v3/projects/67a4f7879e6e46dd9683eeda372d07bf` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.821679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=67a4f7879e6e46dd9683eeda372d07bf)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:28.823836 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:28.857564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:28.858656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.859061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.859441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] PATH_INFO: `/v3/users/3b71cebb993a4f6b82a8145972fb3c04` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.861255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b71cebb993a4f6b82a8145972fb3c04)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:28.864066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:28.913612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:28.927082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67a4f7879e6e46dd9683eeda372d07bf', 'request_id': 'req-11c17c31-8535-4cbd-b344-9ab9cc6917ac'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:28.942306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67a4f7879e6e46dd9683eeda372d07bf', 'request_id': 'req-11c17c31-8535-4cbd-b344-9ab9cc6917ac'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:28.958605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67a4f7879e6e46dd9683eeda372d07bf', 'request_id': 'req-11c17c31-8535-4cbd-b344-9ab9cc6917ac'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:28.973719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] The token cache is being invalidate because project 67a4f7879e6e46dd9683eeda372d07bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:28.974306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:28.975301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:28.977656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b71cebb993a4f6b82a8145972fb3c04'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:28.979553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 911/1815] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:28 2022] DELETE /identity/v3/projects/67a4f7879e6e46dd9683eeda372d07bf => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:28.980895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:28.982950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:28.990171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b71cebb993a4f6b82a8145972fb3c04'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.026632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b71cebb993a4f6b82a8145972fb3c04'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.033044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:29.042568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b71cebb993a4f6b82a8145972fb3c04'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.058120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b71cebb993a4f6b82a8145972fb3c04'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.098786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 905/1816] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:28 2022] DELETE /identity/v3/users/3b71cebb993a4f6b82a8145972fb3c04 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:29.099936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:29.107505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:29.268978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c17c31-8535-4cbd-b344-9ab9cc6917ac admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.305458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8120f-da0a-4a88-8113-c6ec2a408680 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.314015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.315377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.315798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.316721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] PATH_INFO: `/v3/users/245fae7b529c455d9e441004faafe26b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.318703 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=245fae7b529c455d9e441004faafe26b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:29.321465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:29.448579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '245fae7b529c455d9e441004faafe26b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.463677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee96f2-0c92-4241-b770-98b913ea8581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.464405 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee96f2-0c92-4241-b770-98b913ea8581 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.464730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee96f2-0c92-4241-b770-98b913ea8581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.465771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee96f2-0c92-4241-b770-98b913ea8581 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.466770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee96f2-0c92-4241-b770-98b913ea8581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:29.468991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee96f2-0c92-4241-b770-98b913ea8581 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:29.477324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '245fae7b529c455d9e441004faafe26b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.515488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 912/1817] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:28 2022] POST /identity/v3/users => generated 277 bytes in 535 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:29.521702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee96f2-0c92-4241-b770-98b913ea8581 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:29.525953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-00ee96f2-0c92-4241-b770-98b913ea8581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:29.533280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '245fae7b529c455d9e441004faafe26b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.547825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '245fae7b529c455d9e441004faafe26b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.549118 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '245fae7b529c455d9e441004faafe26b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:29.577425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee96f2-0c92-4241-b770-98b913ea8581 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.583971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 906/1818] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:29 2022] DELETE /identity/v3/users/245fae7b529c455d9e441004faafe26b => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:29.679738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f110b0-ce81-4c85-a555-2d6e2b9bb71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.680471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f110b0-ce81-4c85-a555-2d6e2b9bb71a admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.680927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f110b0-ce81-4c85-a555-2d6e2b9bb71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.681273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f110b0-ce81-4c85-a555-2d6e2b9bb71a admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.682114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f110b0-ce81-4c85-a555-2d6e2b9bb71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:29.684565 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f110b0-ce81-4c85-a555-2d6e2b9bb71a admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:29.698287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:29.738806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 913/1819] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:29.739439 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:29.744112 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:29.765355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132b201-226a-4fb6-a866-b194a09e7c6d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.771142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50174e-b14d-44d3-a655-78c31f70c12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.772206 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50174e-b14d-44d3-a655-78c31f70c12b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.772658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50174e-b14d-44d3-a655-78c31f70c12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.773039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50174e-b14d-44d3-a655-78c31f70c12b admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.774443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50174e-b14d-44d3-a655-78c31f70c12b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:29.776448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50174e-b14d-44d3-a655-78c31f70c12b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:29.791566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 907/1820] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:29.797586 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f110b0-ce81-4c85-a555-2d6e2b9bb71a admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:29.820302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f110b0-ce81-4c85-a555-2d6e2b9bb71a admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.827102 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316b192-78b0-4ec9-8e43-f5daea40d12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.828039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316b192-78b0-4ec9-8e43-f5daea40d12c admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.828323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316b192-78b0-4ec9-8e43-f5daea40d12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.828645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316b192-78b0-4ec9-8e43-f5daea40d12c admin admin] PATH_INFO: `/v3/projects/8ba05be1e1da46d395460dc420528e2e/users/253774f720204f9e812b0c5b1f3e1eb1/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.858499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316b192-78b0-4ec9-8e43-f5daea40d12c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba05be1e1da46d395460dc420528e2e, user_id=253774f720204f9e812b0c5b1f3e1eb1, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:29.863089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316b192-78b0-4ec9-8e43-f5daea40d12c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:29.898483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 914/1821] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:29 2022] PUT /identity/v3/projects/8ba05be1e1da46d395460dc420528e2e/users/253774f720204f9e812b0c5b1f3e1eb1/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:29.910614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97947c72-0ae4-41cf-b670-57115925d60c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.911555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97947c72-0ae4-41cf-b670-57115925d60c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.911986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97947c72-0ae4-41cf-b670-57115925d60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.001420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-97947c72-0ae4-41cf-b670-57115925d60c None None] MFA Rules not processed for user `253774f720204f9e812b0c5b1f3e1eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:30.001420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-97947c72-0ae4-41cf-b670-57115925d60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:30.057624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316b192-78b0-4ec9-8e43-f5daea40d12c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.060094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a316b192-78b0-4ec9-8e43-f5daea40d12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:30.247431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:30.284385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316b192-78b0-4ec9-8e43-f5daea40d12c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:30.300353 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:30.303389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.306428 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.307004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] PATH_INFO: `/v3/projects/45642dd64390418c82536819ddbc5b2c` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.312200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] RBAC: Authorizing `identity:delete_project(project_id=45642dd64390418c82536819ddbc5b2c)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:30.320773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:30.403874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 908/1822] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:29 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:30.405580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97947c72-0ae4-41cf-b670-57115925d60c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.482582 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97947c72-0ae4-41cf-b670-57115925d60c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:30.587369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45642dd64390418c82536819ddbc5b2c', 'request_id': 'req-88be1420-649c-4024-a4ec-91760580a92c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:30.626045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45642dd64390418c82536819ddbc5b2c', 'request_id': 'req-88be1420-649c-4024-a4ec-91760580a92c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:30.651747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45642dd64390418c82536819ddbc5b2c', 'request_id': 'req-88be1420-649c-4024-a4ec-91760580a92c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:30.696926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e368dae-ebef-4ee3-8833-c1cb35c4f857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:30.698824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] The token cache is being invalidate because project 45642dd64390418c82536819ddbc5b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:30.699926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:30.702096 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e368dae-ebef-4ee3-8833-c1cb35c4f857 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.702691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:30.704549 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e368dae-ebef-4ee3-8833-c1cb35c4f857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.705253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e368dae-ebef-4ee3-8833-c1cb35c4f857 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.706292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e368dae-ebef-4ee3-8833-c1cb35c4f857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:30.708362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 915/1823] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:30 2022] DELETE /identity/v3/projects/45642dd64390418c82536819ddbc5b2c => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:30.711486 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.715964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e368dae-ebef-4ee3-8833-c1cb35c4f857 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:30.716800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:30.761940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be1420-649c-4024-a4ec-91760580a92c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:30.790990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:30.800986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.802682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.804904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] PATH_INFO: `/v3/users/cc0017a4dfb440828150d1a89b8635aa` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.812096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc0017a4dfb440828150d1a89b8635aa)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:30.828656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:30.833455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 909/1824] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:30 2022] POST /identity/v3/projects => generated 357 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:30.836440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a5bb7-7803-4cc8-822c-f252eca4619f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.841713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a5bb7-7803-4cc8-822c-f252eca4619f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.845999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a5bb7-7803-4cc8-822c-f252eca4619f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.944918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7a5a5bb7-7803-4cc8-822c-f252eca4619f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:30.948652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7a5a5bb7-7803-4cc8-822c-f252eca4619f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:30.978083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc0017a4dfb440828150d1a89b8635aa'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.004584 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc0017a4dfb440828150d1a89b8635aa'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.005183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc0017a4dfb440828150d1a89b8635aa'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.022127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:31.027980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc0017a4dfb440828150d1a89b8635aa'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.028870 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc0017a4dfb440828150d1a89b8635aa'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:31.092420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 916/1825] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:30 2022] DELETE /identity/v3/users/cc0017a4dfb440828150d1a89b8635aa => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:31.094014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.098911 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:31.215830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b79f5-5da7-4fce-81ff-3d5864eef00d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.229430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62443326-367b-4567-9ada-105635954744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.230216 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62443326-367b-4567-9ada-105635954744 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.230614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62443326-367b-4567-9ada-105635954744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.230992 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62443326-367b-4567-9ada-105635954744 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.233582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62443326-367b-4567-9ada-105635954744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.234616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62443326-367b-4567-9ada-105635954744 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.272352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 917/1826] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:31 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:31.273891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 910/1827] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:31.274030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc278e-d17e-4d60-bc3f-8e1d4e1827fd None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.274197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc278e-d17e-4d60-bc3f-8e1d4e1827fd None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.275480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a5bb7-7803-4cc8-822c-f252eca4619f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.276187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc278e-d17e-4d60-bc3f-8e1d4e1827fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.315219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-68dc278e-d17e-4d60-bc3f-8e1d4e1827fd None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:31.316750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-68dc278e-d17e-4d60-bc3f-8e1d4e1827fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:31.331014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 918/1828] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:31.331279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11010f20-0169-4d65-9fcd-2b7d87a01424 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.331463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11010f20-0169-4d65-9fcd-2b7d87a01424 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.331799 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11010f20-0169-4d65-9fcd-2b7d87a01424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.355329 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a5bb7-7803-4cc8-822c-f252eca4619f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.357469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-11010f20-0169-4d65-9fcd-2b7d87a01424 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:31.358963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-11010f20-0169-4d65-9fcd-2b7d87a01424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:31.374337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1e415-685d-4b39-aa75-0dbe13a05008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.376428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1e415-685d-4b39-aa75-0dbe13a05008 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.377038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 919/1829] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:31.378061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1e415-685d-4b39-aa75-0dbe13a05008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.378823 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1e415-685d-4b39-aa75-0dbe13a05008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.381086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1e415-685d-4b39-aa75-0dbe13a05008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.383753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11010f20-0169-4d65-9fcd-2b7d87a01424 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.387339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1e415-685d-4b39-aa75-0dbe13a05008 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.412788 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 911/1830] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:31.414407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11010f20-0169-4d65-9fcd-2b7d87a01424 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.423691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1e415-685d-4b39-aa75-0dbe13a05008 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.431281 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679dc69-1eb2-4841-bd1d-01355b8a8b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.435199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679dc69-1eb2-4841-bd1d-01355b8a8b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.436123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679dc69-1eb2-4841-bd1d-01355b8a8b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.437093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679dc69-1eb2-4841-bd1d-01355b8a8b8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.439572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679dc69-1eb2-4841-bd1d-01355b8a8b8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:31.440068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679dc69-1eb2-4841-bd1d-01355b8a8b8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.446743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679dc69-1eb2-4841-bd1d-01355b8a8b8a admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.457754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1e415-685d-4b39-aa75-0dbe13a05008 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.471092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de13d8b-439c-4e48-bf85-727b9c2e88d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.474088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de13d8b-439c-4e48-bf85-727b9c2e88d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.474479 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de13d8b-439c-4e48-bf85-727b9c2e88d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.475343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de13d8b-439c-4e48-bf85-727b9c2e88d1 admin admin] PATH_INFO: `/v3/projects/17725f01b84242dba92a8727d6c9df53/users/9cf001c0c1584bfb83287e5ef5500324/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.489346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 920/1831] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:31.490341 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679dc69-1eb2-4841-bd1d-01355b8a8b8a admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.502330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8679dc69-1eb2-4841-bd1d-01355b8a8b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:31.540742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de13d8b-439c-4e48-bf85-727b9c2e88d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17725f01b84242dba92a8727d6c9df53, user_id=9cf001c0c1584bfb83287e5ef5500324, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.569177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de13d8b-439c-4e48-bf85-727b9c2e88d1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.639357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 912/1832] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:31 2022] PUT /identity/v3/projects/17725f01b84242dba92a8727d6c9df53/users/9cf001c0c1584bfb83287e5ef5500324/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:31.645701 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de13d8b-439c-4e48-bf85-727b9c2e88d1 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.718705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:31.743839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de13d8b-439c-4e48-bf85-727b9c2e88d1 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.749008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679dc69-1eb2-4841-bd1d-01355b8a8b8a admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.749915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733c531-c8d1-44f8-9f0c-439f7876c223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.750499 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733c531-c8d1-44f8-9f0c-439f7876c223 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.750882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733c531-c8d1-44f8-9f0c-439f7876c223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.751400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733c531-c8d1-44f8-9f0c-439f7876c223 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.752221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733c531-c8d1-44f8-9f0c-439f7876c223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.755753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733c531-c8d1-44f8-9f0c-439f7876c223 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.829446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 913/1833] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:31 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:31.835306 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733c531-c8d1-44f8-9f0c-439f7876c223 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.850608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffe7c9-0e42-4548-b4f3-55531e5669e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.851300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffe7c9-0e42-4548-b4f3-55531e5669e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.851873 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffe7c9-0e42-4548-b4f3-55531e5669e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.852148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffe7c9-0e42-4548-b4f3-55531e5669e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.865203 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffe7c9-0e42-4548-b4f3-55531e5669e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.867946 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffe7c9-0e42-4548-b4f3-55531e5669e7 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.875943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:31.888655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 921/1834] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:31.891162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffe7c9-0e42-4548-b4f3-55531e5669e7 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.917151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffe7c9-0e42-4548-b4f3-55531e5669e7 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:32.113573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30791f25-7045-4eab-917f-107b003d7b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:32.117429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30791f25-7045-4eab-917f-107b003d7b84 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.117983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30791f25-7045-4eab-917f-107b003d7b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.118462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30791f25-7045-4eab-917f-107b003d7b84 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.120213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30791f25-7045-4eab-917f-107b003d7b84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:32.128086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30791f25-7045-4eab-917f-107b003d7b84 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:32.191154 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 922/1835] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:31 2022] POST /identity/v3/users => generated 291 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:32.198383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30791f25-7045-4eab-917f-107b003d7b84 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:32.219642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30791f25-7045-4eab-917f-107b003d7b84 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:32.238206 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ba03c-7532-40f0-8189-345461e09e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:32.239135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ba03c-7532-40f0-8189-345461e09e63 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.239596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ba03c-7532-40f0-8189-345461e09e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.240029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ba03c-7532-40f0-8189-345461e09e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.241523 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ba03c-7532-40f0-8189-345461e09e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:32.242320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733c531-c8d1-44f8-9f0c-439f7876c223 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:32.244221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24ba03c-7532-40f0-8189-345461e09e63 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:32.264894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cd99d-8b97-4fc5-a87a-adfac7e26351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:32.267924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cd99d-8b97-4fc5-a87a-adfac7e26351 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.269858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cd99d-8b97-4fc5-a87a-adfac7e26351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.270394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cd99d-8b97-4fc5-a87a-adfac7e26351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.271792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2cd99d-8b97-4fc5-a87a-adfac7e26351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:32.285232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2cd99d-8b97-4fc5-a87a-adfac7e26351 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:32.311043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 923/1836] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:32.316617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ba03c-7532-40f0-8189-345461e09e63 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:32.342399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ba03c-7532-40f0-8189-345461e09e63 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:32.358425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679013b-6346-4ecf-bd22-531837f521e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:32.359610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679013b-6346-4ecf-bd22-531837f521e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.360174 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679013b-6346-4ecf-bd22-531837f521e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.360864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679013b-6346-4ecf-bd22-531837f521e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.362040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679013b-6346-4ecf-bd22-531837f521e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:32.368680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679013b-6346-4ecf-bd22-531837f521e4 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:32.408097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 924/1837] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:32.409762 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679013b-6346-4ecf-bd22-531837f521e4 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:32.426964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 914/1838] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 596 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:32.434166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cd99d-8b97-4fc5-a87a-adfac7e26351 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:32.439392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679013b-6346-4ecf-bd22-531837f521e4 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:32.455202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9308662-dfe5-4242-93fe-03d6042a31ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:32.458981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9308662-dfe5-4242-93fe-03d6042a31ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.464056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9308662-dfe5-4242-93fe-03d6042a31ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.464056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9308662-dfe5-4242-93fe-03d6042a31ea admin admin] PATH_INFO: `/v3/projects/8ced3c5aa3994189a43d5eb81dd9d657/users/561ef03eecbf447f9a5c94f3f0851c75/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.509095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9308662-dfe5-4242-93fe-03d6042a31ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ced3c5aa3994189a43d5eb81dd9d657, user_id=561ef03eecbf447f9a5c94f3f0851c75, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:32.517845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9308662-dfe5-4242-93fe-03d6042a31ea admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:32.522568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cd99d-8b97-4fc5-a87a-adfac7e26351 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:32.552653 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 925/1839] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:32 2022] PUT /identity/v3/projects/8ced3c5aa3994189a43d5eb81dd9d657/users/561ef03eecbf447f9a5c94f3f0851c75/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:32.576657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03faa59-a092-43a0-b808-bf94f0487600 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.577491 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03faa59-a092-43a0-b808-bf94f0487600 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.577945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03faa59-a092-43a0-b808-bf94f0487600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.594532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baebf50-419b-46c4-b2c3-bc0934121b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:32.601480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baebf50-419b-46c4-b2c3-bc0934121b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.603517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baebf50-419b-46c4-b2c3-bc0934121b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.604410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baebf50-419b-46c4-b2c3-bc0934121b62 admin admin] PATH_INFO: `/v3/projects/17725f01b84242dba92a8727d6c9df53/users/9cf001c0c1584bfb83287e5ef5500324/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.613462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baebf50-419b-46c4-b2c3-bc0934121b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17725f01b84242dba92a8727d6c9df53, user_id=9cf001c0c1584bfb83287e5ef5500324, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:32.619345 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baebf50-419b-46c4-b2c3-bc0934121b62 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:32.635368 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 915/1840] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:32 2022] PUT /identity/v3/projects/17725f01b84242dba92a8727d6c9df53/users/9cf001c0c1584bfb83287e5ef5500324/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:32.660565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ce0df-dc99-41d5-8cb8-f98b2f80d5e6 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.661224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ce0df-dc99-41d5-8cb8-f98b2f80d5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.662135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ce0df-dc99-41d5-8cb8-f98b2f80d5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.746920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-b03faa59-a092-43a0-b808-bf94f0487600 None None] MFA Rules not processed for user `561ef03eecbf447f9a5c94f3f0851c75`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:32.749482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b03faa59-a092-43a0-b808-bf94f0487600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:32.769306 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-363ce0df-dc99-41d5-8cb8-f98b2f80d5e6 None None] MFA Rules not processed for user `9cf001c0c1584bfb83287e5ef5500324`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:32.775170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-363ce0df-dc99-41d5-8cb8-f98b2f80d5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:33.030754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 926/1841] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:33.032326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03faa59-a092-43a0-b808-bf94f0487600 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:33.043874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b03faa59-a092-43a0-b808-bf94f0487600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:33.111403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 916/1842] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:32 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:33.114541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ce0df-dc99-41d5-8cb8-f98b2f80d5e6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:33.209485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:33.320498 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03faa59-a092-43a0-b808-bf94f0487600 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:33.346448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:33.346825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:33.347949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:33.348443 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:33.348946 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] PATH_INFO: `/v3/projects/91f4cd559ae5435c852efe424bf028ca` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:33.362020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f4cd559ae5435c852efe424bf028ca)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:33.365662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:33.473679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ce0df-dc99-41d5-8cb8-f98b2f80d5e6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:33.511162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dc04e-7db8-414f-8c76-29c34a627853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:33.512943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dc04e-7db8-414f-8c76-29c34a627853 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:33.513437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dc04e-7db8-414f-8c76-29c34a627853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:33.513929 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dc04e-7db8-414f-8c76-29c34a627853 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:33.515326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385dc04e-7db8-414f-8c76-29c34a627853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:33.530898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385dc04e-7db8-414f-8c76-29c34a627853 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:33.611244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f4cd559ae5435c852efe424bf028ca', 'request_id': 'req-eab1311c-de4d-48d6-8e50-45b42de5d774'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:33.645784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f4cd559ae5435c852efe424bf028ca', 'request_id': 'req-eab1311c-de4d-48d6-8e50-45b42de5d774'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:33.668895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f4cd559ae5435c852efe424bf028ca', 'request_id': 'req-eab1311c-de4d-48d6-8e50-45b42de5d774'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:33.679143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 917/1843] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:33 2022] POST /identity/v3/projects => generated 379 bytes in 560 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:33.680863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dc04e-7db8-414f-8c76-29c34a627853 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:33.686072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-385dc04e-7db8-414f-8c76-29c34a627853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:33.727965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] The token cache is being invalidate because project 91f4cd559ae5435c852efe424bf028ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:33.728678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:33.736598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:33.740049 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 927/1844] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:33 2022] DELETE /identity/v3/projects/91f4cd559ae5435c852efe424bf028ca => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:33.743710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:33.751685 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:34.008566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dc04e-7db8-414f-8c76-29c34a627853 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.024070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692a907-ff1b-45c3-8b5d-d85be5a81f84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.027756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692a907-ff1b-45c3-8b5d-d85be5a81f84 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.028258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692a907-ff1b-45c3-8b5d-d85be5a81f84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.041081 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692a907-ff1b-45c3-8b5d-d85be5a81f84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.047307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7692a907-ff1b-45c3-8b5d-d85be5a81f84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:34.110878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1311c-de4d-48d6-8e50-45b42de5d774 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.265559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692a907-ff1b-45c3-8b5d-d85be5a81f84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.267145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692a907-ff1b-45c3-8b5d-d85be5a81f84 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.295579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 918/1845] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:33 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:34.298849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e67dce-4ae9-4708-963f-d2d574815cc3 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.300615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e67dce-4ae9-4708-963f-d2d574815cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.301149 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e67dce-4ae9-4708-963f-d2d574815cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.366487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-61e67dce-4ae9-4708-963f-d2d574815cc3 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:34.374146 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af4f8b-d10e-4d1f-b6ae-1771a684b4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.376013 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af4f8b-d10e-4d1f-b6ae-1771a684b4d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.376753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af4f8b-d10e-4d1f-b6ae-1771a684b4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.377311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-61e67dce-4ae9-4708-963f-d2d574815cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:34.377492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af4f8b-d10e-4d1f-b6ae-1771a684b4d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.378724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af4f8b-d10e-4d1f-b6ae-1771a684b4d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.381782 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af4f8b-d10e-4d1f-b6ae-1771a684b4d6 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.426352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:34.439725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 919/1846] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:34.442611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816584b-d149-4928-8d9b-865486a1118f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.443881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816584b-d149-4928-8d9b-865486a1118f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.445449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816584b-d149-4928-8d9b-865486a1118f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.450268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 928/1847] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:33 2022] POST /identity/v3/users => generated 296 bytes in 708 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:34.453244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af4f8b-d10e-4d1f-b6ae-1771a684b4d6 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.474241 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af4f8b-d10e-4d1f-b6ae-1771a684b4d6 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.481171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b67d21-e9cc-45c1-96c6-9a37b0a0cc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.482137 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b67d21-e9cc-45c1-96c6-9a37b0a0cc39 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.482567 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b67d21-e9cc-45c1-96c6-9a37b0a0cc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.483317 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b67d21-e9cc-45c1-96c6-9a37b0a0cc39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.484418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b67d21-e9cc-45c1-96c6-9a37b0a0cc39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.486373 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b67d21-e9cc-45c1-96c6-9a37b0a0cc39 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.489604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8816584b-d149-4928-8d9b-865486a1118f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:34.490867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8816584b-d149-4928-8d9b-865486a1118f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:34.499179 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 929/1848] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:34.505018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b67d21-e9cc-45c1-96c6-9a37b0a0cc39 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.511269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 920/1849] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:34.513644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816584b-d149-4928-8d9b-865486a1118f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.516300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8816584b-d149-4928-8d9b-865486a1118f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:34.521122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b67d21-e9cc-45c1-96c6-9a37b0a0cc39 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.526978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725ee51-6427-4425-9736-ab9742cb4ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.528612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725ee51-6427-4425-9736-ab9742cb4ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.528884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725ee51-6427-4425-9736-ab9742cb4ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.529169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725ee51-6427-4425-9736-ab9742cb4ce1 admin admin] PATH_INFO: `/v3/projects/2a2657bc1d3c44b1b73af9273d80e032/users/2daef1434fff40b8b5732294a3a26258/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.539126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816584b-d149-4928-8d9b-865486a1118f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.545081 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e7632-aa94-4436-8c05-66f1ffecbb66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.546362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e7632-aa94-4436-8c05-66f1ffecbb66 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.546917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e7632-aa94-4436-8c05-66f1ffecbb66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.547392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e7632-aa94-4436-8c05-66f1ffecbb66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.548258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725ee51-6427-4425-9736-ab9742cb4ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2657bc1d3c44b1b73af9273d80e032, user_id=2daef1434fff40b8b5732294a3a26258, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.551075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725ee51-6427-4425-9736-ab9742cb4ce1 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.552204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8f2e7632-aa94-4436-8c05-66f1ffecbb66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:34.571701 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 930/1850] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:34 2022] PUT /identity/v3/projects/2a2657bc1d3c44b1b73af9273d80e032/users/2daef1434fff40b8b5732294a3a26258/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:34.572467 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725ee51-6427-4425-9736-ab9742cb4ce1 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.659500 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:34.663224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725ee51-6427-4425-9736-ab9742cb4ce1 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.665010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e7632-aa94-4436-8c05-66f1ffecbb66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.668964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e7632-aa94-4436-8c05-66f1ffecbb66 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.670471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158817c-4129-40da-ac30-927ef99f4baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.671261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158817c-4129-40da-ac30-927ef99f4baa admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.671677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158817c-4129-40da-ac30-927ef99f4baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.672050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158817c-4129-40da-ac30-927ef99f4baa admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.672973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158817c-4129-40da-ac30-927ef99f4baa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:34.673348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158817c-4129-40da-ac30-927ef99f4baa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.675122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158817c-4129-40da-ac30-927ef99f4baa admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.685285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 921/1851] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:34 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:34.686979 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e7632-aa94-4436-8c05-66f1ffecbb66 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.700886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 931/1852] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:34.706450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158817c-4129-40da-ac30-927ef99f4baa admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.717837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e7632-aa94-4436-8c05-66f1ffecbb66 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.732746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158817c-4129-40da-ac30-927ef99f4baa admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.734907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89345e27-8900-41b1-b0b4-9fd440487e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.736391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89345e27-8900-41b1-b0b4-9fd440487e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.736533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89345e27-8900-41b1-b0b4-9fd440487e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.737091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89345e27-8900-41b1-b0b4-9fd440487e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.739459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89345e27-8900-41b1-b0b4-9fd440487e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.742492 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89345e27-8900-41b1-b0b4-9fd440487e3a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.752653 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728caf1c-14ee-464b-98fb-247817a5f19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.754982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728caf1c-14ee-464b-98fb-247817a5f19f admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.755518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728caf1c-14ee-464b-98fb-247817a5f19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.755985 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728caf1c-14ee-464b-98fb-247817a5f19f admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.757898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728caf1c-14ee-464b-98fb-247817a5f19f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.761029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728caf1c-14ee-464b-98fb-247817a5f19f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.772019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 922/1853] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:34.776030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89345e27-8900-41b1-b0b4-9fd440487e3a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.796470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89345e27-8900-41b1-b0b4-9fd440487e3a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.809782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e8efd-d109-42e3-b9c1-fbe08fcde69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.810429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e8efd-d109-42e3-b9c1-fbe08fcde69b admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.810844 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e8efd-d109-42e3-b9c1-fbe08fcde69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.811300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e8efd-d109-42e3-b9c1-fbe08fcde69b admin admin] PATH_INFO: `/v3/projects/2a2657bc1d3c44b1b73af9273d80e032/users/2daef1434fff40b8b5732294a3a26258/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.815210 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e8efd-d109-42e3-b9c1-fbe08fcde69b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2657bc1d3c44b1b73af9273d80e032, user_id=2daef1434fff40b8b5732294a3a26258, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.817201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 932/1854] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:34 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:34.820209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728caf1c-14ee-464b-98fb-247817a5f19f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.823976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e8efd-d109-42e3-b9c1-fbe08fcde69b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.866428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 923/1855] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:34 2022] PUT /identity/v3/projects/2a2657bc1d3c44b1b73af9273d80e032/users/2daef1434fff40b8b5732294a3a26258/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:34.877413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806b91f-a212-4049-8df9-e15dca91cfa7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.877908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806b91f-a212-4049-8df9-e15dca91cfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.878199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806b91f-a212-4049-8df9-e15dca91cfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.017137 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7806b91f-a212-4049-8df9-e15dca91cfa7 None None] MFA Rules not processed for user `2daef1434fff40b8b5732294a3a26258`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:35.020361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7806b91f-a212-4049-8df9-e15dca91cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:35.027214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728caf1c-14ee-464b-98fb-247817a5f19f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.159400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:35.206295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba89ece-181a-4d0b-bf77-87cbec213ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.207706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba89ece-181a-4d0b-bf77-87cbec213ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.208295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba89ece-181a-4d0b-bf77-87cbec213ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.208934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba89ece-181a-4d0b-bf77-87cbec213ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.210784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba89ece-181a-4d0b-bf77-87cbec213ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.214773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba89ece-181a-4d0b-bf77-87cbec213ff9 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.238773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 924/1856] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:35.256890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806b91f-a212-4049-8df9-e15dca91cfa7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.267207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 933/1857] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:34 2022] POST /identity/v3/users => generated 294 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:35.269172 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba89ece-181a-4d0b-bf77-87cbec213ff9 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.281712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7806b91f-a212-4049-8df9-e15dca91cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:35.298962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba89ece-181a-4d0b-bf77-87cbec213ff9 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.316847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69418fc-9226-4cb2-b164-959313f749b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.318118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69418fc-9226-4cb2-b164-959313f749b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.331086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69418fc-9226-4cb2-b164-959313f749b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.331631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69418fc-9226-4cb2-b164-959313f749b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.333128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69418fc-9226-4cb2-b164-959313f749b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.336516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69418fc-9226-4cb2-b164-959313f749b2 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.385107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 934/1858] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:35.392304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69418fc-9226-4cb2-b164-959313f749b2 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.402076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d69418fc-9226-4cb2-b164-959313f749b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:35.423283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806b91f-a212-4049-8df9-e15dca91cfa7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.429456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79298fa6-e898-4f68-a3f3-ae977ba8fc7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.430476 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79298fa6-e898-4f68-a3f3-ae977ba8fc7b service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.430980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79298fa6-e898-4f68-a3f3-ae977ba8fc7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.431565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79298fa6-e898-4f68-a3f3-ae977ba8fc7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.445006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69418fc-9226-4cb2-b164-959313f749b2 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.447953 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79298fa6-e898-4f68-a3f3-ae977ba8fc7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.450798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79298fa6-e898-4f68-a3f3-ae977ba8fc7b service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.460098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.461047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.461338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.461669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] PATH_INFO: `/v3/projects/a4fb92f1e72f4228b7bf298aac07414f` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.464429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4fb92f1e72f4228b7bf298aac07414f)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.467231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.486403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 925/1859] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:35.490772 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79298fa6-e898-4f68-a3f3-ae977ba8fc7b service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.539850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79298fa6-e898-4f68-a3f3-ae977ba8fc7b service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.560711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3d892-af64-40cd-9644-0f238debf848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.562134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3d892-af64-40cd-9644-0f238debf848 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.562755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3d892-af64-40cd-9644-0f238debf848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.563292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3d892-af64-40cd-9644-0f238debf848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.565187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3d892-af64-40cd-9644-0f238debf848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.576020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3d892-af64-40cd-9644-0f238debf848 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.632637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 926/1860] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:35.642436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3d892-af64-40cd-9644-0f238debf848 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.669438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4fb92f1e72f4228b7bf298aac07414f', 'request_id': 'req-594ce739-4492-4901-b0f1-b9715decf086'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:35.692024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3d892-af64-40cd-9644-0f238debf848 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.705891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4fb92f1e72f4228b7bf298aac07414f', 'request_id': 'req-594ce739-4492-4901-b0f1-b9715decf086'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:35.727418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4fb92f1e72f4228b7bf298aac07414f', 'request_id': 'req-594ce739-4492-4901-b0f1-b9715decf086'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:35.777719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] The token cache is being invalidate because project a4fb92f1e72f4228b7bf298aac07414f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:35.778684 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:35.783221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:35.795764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 935/1861] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:35 2022] DELETE /identity/v3/projects/a4fb92f1e72f4228b7bf298aac07414f => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:35.797651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.802000 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:35.869727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:35.927611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd5ace-f893-4ed1-88af-39caa6c5d27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.929153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd5ace-f893-4ed1-88af-39caa6c5d27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.929791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd5ace-f893-4ed1-88af-39caa6c5d27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.930105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd5ace-f893-4ed1-88af-39caa6c5d27f admin admin] PATH_INFO: `/v3/projects/e488b9697cfd497e945cf37d6691548c/users/52ee3de3c8774fb99da017cda4a1e7aa/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.958348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ce739-4492-4901-b0f1-b9715decf086 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.965504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.966499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.966778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.967087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] PATH_INFO: `/v3/users/501569bb389f42e1893c9e993aa8f837` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.968642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] RBAC: Authorizing `identity:delete_user(user_id=501569bb389f42e1893c9e993aa8f837)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.969266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd5ace-f893-4ed1-88af-39caa6c5d27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e488b9697cfd497e945cf37d6691548c, user_id=52ee3de3c8774fb99da017cda4a1e7aa, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.970723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.974695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd5ace-f893-4ed1-88af-39caa6c5d27f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.004022 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 927/1862] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:35 2022] PUT /identity/v3/projects/e488b9697cfd497e945cf37d6691548c/users/52ee3de3c8774fb99da017cda4a1e7aa/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:36.018470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8ddd4-f3f2-4661-ab03-09df1c6d81e8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.018898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8ddd4-f3f2-4661-ab03-09df1c6d81e8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.019344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8ddd4-f3f2-4661-ab03-09df1c6d81e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.071815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-28a8ddd4-f3f2-4661-ab03-09df1c6d81e8 None None] MFA Rules not processed for user `52ee3de3c8774fb99da017cda4a1e7aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:36.073763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-28a8ddd4-f3f2-4661-ab03-09df1c6d81e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:36.109134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '501569bb389f42e1893c9e993aa8f837'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:36.113376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:36.128887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '501569bb389f42e1893c9e993aa8f837'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:36.130517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '501569bb389f42e1893c9e993aa8f837'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:36.148327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '501569bb389f42e1893c9e993aa8f837'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:36.149803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '501569bb389f42e1893c9e993aa8f837'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:36.190052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 928/1863] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:36.197049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8ddd4-f3f2-4661-ab03-09df1c6d81e8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.200107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 936/1864] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:35 2022] DELETE /identity/v3/users/501569bb389f42e1893c9e993aa8f837 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:36.201230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-28a8ddd4-f3f2-4661-ab03-09df1c6d81e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:36.219330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.224823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:36.377373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8ddd4-f3f2-4661-ab03-09df1c6d81e8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.389044 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514deaf-d259-4379-8d7c-496dd7db231d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.404659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b6574-01a3-4dad-8301-fbac87315c27 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2daef1434fff40b8b5732294a3a26258', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a2657bc1d3c44b1b73af9273d80e032', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.406139 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b6574-01a3-4dad-8301-fbac87315c27 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.406755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b6574-01a3-4dad-8301-fbac87315c27 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.407362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b6574-01a3-4dad-8301-fbac87315c27 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.409145 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b6574-01a3-4dad-8301-fbac87315c27 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:36.409804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b6574-01a3-4dad-8301-fbac87315c27 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:36.414823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b6574-01a3-4dad-8301-fbac87315c27 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.443513 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 937/1865] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:36.452979 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b6574-01a3-4dad-8301-fbac87315c27 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.478906 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b6574-01a3-4dad-8301-fbac87315c27 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.488401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7d985-12ad-4c7e-b281-962892d37a3f tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2daef1434fff40b8b5732294a3a26258', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a2657bc1d3c44b1b73af9273d80e032', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.489529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7d985-12ad-4c7e-b281-962892d37a3f tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.489859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7d985-12ad-4c7e-b281-962892d37a3f tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.490246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7d985-12ad-4c7e-b281-962892d37a3f tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.490993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7d985-12ad-4c7e-b281-962892d37a3f tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:36.494635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7d985-12ad-4c7e-b281-962892d37a3f tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.537554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea06147-0aea-4301-9bc3-9839e6b8b09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.541934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 938/1866] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:36 2022] POST /identity/v3/projects => generated 355 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:36.542115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea06147-0aea-4301-9bc3-9839e6b8b09c admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.542115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea06147-0aea-4301-9bc3-9839e6b8b09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.542115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea06147-0aea-4301-9bc3-9839e6b8b09c admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.542360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea06147-0aea-4301-9bc3-9839e6b8b09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:36.544638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7d985-12ad-4c7e-b281-962892d37a3f tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.545118 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea06147-0aea-4301-9bc3-9839e6b8b09c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.566341 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:36.596551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 929/1867] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:36 2022] POST /identity/v3/projects => generated 385 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:36.600189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea06147-0aea-4301-9bc3-9839e6b8b09c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.655641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7d985-12ad-4c7e-b281-962892d37a3f tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.694741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea06147-0aea-4301-9bc3-9839e6b8b09c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.699990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd9efc-4043-4d40-961b-9caee9285748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.700704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd9efc-4043-4d40-961b-9caee9285748 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.700990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd9efc-4043-4d40-961b-9caee9285748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.701258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd9efc-4043-4d40-961b-9caee9285748 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.701965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd9efc-4043-4d40-961b-9caee9285748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:36.703910 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd9efc-4043-4d40-961b-9caee9285748 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.711854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:36.729456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 930/1868] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:36 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:36.731303 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd9efc-4043-4d40-961b-9caee9285748 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.736115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ead9e-ae4c-46fc-9df1-e55faeec43fe tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2daef1434fff40b8b5732294a3a26258', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a2657bc1d3c44b1b73af9273d80e032', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.736848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ead9e-ae4c-46fc-9df1-e55faeec43fe tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.737143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ead9e-ae4c-46fc-9df1-e55faeec43fe tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.737625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ead9e-ae4c-46fc-9df1-e55faeec43fe tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.738397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ead9e-ae4c-46fc-9df1-e55faeec43fe tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:36.738720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ead9e-ae4c-46fc-9df1-e55faeec43fe tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:36.741630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ead9e-ae4c-46fc-9df1-e55faeec43fe tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.746716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd9efc-4043-4d40-961b-9caee9285748 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.760533 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 939/1869] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:36.761367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525d6a2-b302-4180-92cd-ca2b9a25289f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.762483 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525d6a2-b302-4180-92cd-ca2b9a25289f admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.762822 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525d6a2-b302-4180-92cd-ca2b9a25289f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.763093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525d6a2-b302-4180-92cd-ca2b9a25289f admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.764299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525d6a2-b302-4180-92cd-ca2b9a25289f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:36.765584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525d6a2-b302-4180-92cd-ca2b9a25289f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.776892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 931/1870] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:36.780972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ead9e-ae4c-46fc-9df1-e55faeec43fe tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.792253 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ead9e-ae4c-46fc-9df1-e55faeec43fe tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.804410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fce922-c879-4e74-b44e-a1dab60acc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.805423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fce922-c879-4e74-b44e-a1dab60acc74 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.805770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fce922-c879-4e74-b44e-a1dab60acc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.806157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fce922-c879-4e74-b44e-a1dab60acc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.807275 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fce922-c879-4e74-b44e-a1dab60acc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:36.809875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fce922-c879-4e74-b44e-a1dab60acc74 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.811214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525d6a2-b302-4180-92cd-ca2b9a25289f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.830510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525d6a2-b302-4180-92cd-ca2b9a25289f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.833090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 940/1871] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:36.833990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fce922-c879-4e74-b44e-a1dab60acc74 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.835399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2daef1434fff40b8b5732294a3a26258', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a2657bc1d3c44b1b73af9273d80e032', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.835898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c0fce922-c879-4e74-b44e-a1dab60acc74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:36.836503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.836888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.837265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] PATH_INFO: `/v3/projects/208dd0095bb742d5b5c5a2957d4ce2df` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.838571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: Authorizing `identity:delete_project(project_id=208dd0095bb742d5b5c5a2957d4ce2df)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:36.841732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.858235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fce922-c879-4e74-b44e-a1dab60acc74 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.864702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb89801-03a1-4d2e-be84-8843abc09ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.865962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb89801-03a1-4d2e-be84-8843abc09ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.866406 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb89801-03a1-4d2e-be84-8843abc09ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.866972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb89801-03a1-4d2e-be84-8843abc09ea9 admin admin] PATH_INFO: `/v3/projects/ae0f5817de184b6797186f21458b5176` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.873672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb89801-03a1-4d2e-be84-8843abc09ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae0f5817de184b6797186f21458b5176)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:36.876118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb89801-03a1-4d2e-be84-8843abc09ea9 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.924579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '208dd0095bb742d5b5c5a2957d4ce2df', 'request_id': 'req-ec45c92a-60c3-4b98-8c41-a97d3a959e17'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:36.931486 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '208dd0095bb742d5b5c5a2957d4ce2df', 'request_id': 'req-ec45c92a-60c3-4b98-8c41-a97d3a959e17'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:36.942247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '208dd0095bb742d5b5c5a2957d4ce2df', 'request_id': 'req-ec45c92a-60c3-4b98-8c41-a97d3a959e17'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:36.960713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] The token cache is being invalidate because project 208dd0095bb742d5b5c5a2957d4ce2df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:36.961114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:36.961895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:36.965610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ebb89801-03a1-4d2e-be84-8843abc09ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae0f5817de184b6797186f21458b5176', 'request_id': 'req-ebb89801-03a1-4d2e-be84-8843abc09ea9'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:36.968062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 932/1872] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:36 2022] DELETE /identity/v3/projects/208dd0095bb742d5b5c5a2957d4ce2df => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:36.969218 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.970728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:36.982244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ebb89801-03a1-4d2e-be84-8843abc09ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae0f5817de184b6797186f21458b5176', 'request_id': 'req-ebb89801-03a1-4d2e-be84-8843abc09ea9'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:36.990857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ebb89801-03a1-4d2e-be84-8843abc09ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae0f5817de184b6797186f21458b5176', 'request_id': 'req-ebb89801-03a1-4d2e-be84-8843abc09ea9'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:37.027812 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ebb89801-03a1-4d2e-be84-8843abc09ea9 admin admin] The token cache is being invalidate because project ae0f5817de184b6797186f21458b5176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:37.027812 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ebb89801-03a1-4d2e-be84-8843abc09ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:37.028151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ebb89801-03a1-4d2e-be84-8843abc09ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:37.032578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 941/1873] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:36 2022] DELETE /identity/v3/projects/ae0f5817de184b6797186f21458b5176 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:37.034634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b9558-0e6d-4803-85d3-0e2285f1d939 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.034933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b9558-0e6d-4803-85d3-0e2285f1d939 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.035243 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b9558-0e6d-4803-85d3-0e2285f1d939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.060270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-661b9558-0e6d-4803-85d3-0e2285f1d939 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:37.062179 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-661b9558-0e6d-4803-85d3-0e2285f1d939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:37.101349 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:37.115655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45c92a-60c3-4b98-8c41-a97d3a959e17 tempest-VolumeQuotaClassesTest-459568018 tempest-VolumeQuotaClassesTest-459568018-project-admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.174017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 942/1874] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:37.183938 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527221b-2882-4922-9a82-5f39fabe1949 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.184711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527221b-2882-4922-9a82-5f39fabe1949 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.184711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527221b-2882-4922-9a82-5f39fabe1949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.222636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-8527221b-2882-4922-9a82-5f39fabe1949 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:37.225868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8527221b-2882-4922-9a82-5f39fabe1949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:37.233433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eaef1-893d-425c-a965-d1f9d8576d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.234681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eaef1-893d-425c-a965-d1f9d8576d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.235189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eaef1-893d-425c-a965-d1f9d8576d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.235619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eaef1-893d-425c-a965-d1f9d8576d14 admin admin] PATH_INFO: `/v3/projects/1cb0b668382d4dcb8c7720178017c159/users/1bfb479e9e1e482e9c4bebda51edfe65/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.241048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 943/1875] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:37.243378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527221b-2882-4922-9a82-5f39fabe1949 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.263872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527221b-2882-4922-9a82-5f39fabe1949 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.277740 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eaef1-893d-425c-a965-d1f9d8576d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cb0b668382d4dcb8c7720178017c159, user_id=1bfb479e9e1e482e9c4bebda51edfe65, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:37.279360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7760c-4160-4450-9a86-95769af55227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.279958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7760c-4160-4450-9a86-95769af55227 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.280319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7760c-4160-4450-9a86-95769af55227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.280718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7760c-4160-4450-9a86-95769af55227 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.281540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7760c-4160-4450-9a86-95769af55227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:37.281967 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7760c-4160-4450-9a86-95769af55227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:37.282774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eaef1-893d-425c-a965-d1f9d8576d14 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:37.283733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7760c-4160-4450-9a86-95769af55227 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:37.300840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 944/1876] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:37.312540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7760c-4160-4450-9a86-95769af55227 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.313506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 933/1877] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:36 2022] PUT /identity/v3/projects/1cb0b668382d4dcb8c7720178017c159/users/1bfb479e9e1e482e9c4bebda51edfe65/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:37.319488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475310d1-058f-40d7-89d5-92941a8cebc9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.320706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475310d1-058f-40d7-89d5-92941a8cebc9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.321064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475310d1-058f-40d7-89d5-92941a8cebc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.338790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7760c-4160-4450-9a86-95769af55227 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.366974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-475310d1-058f-40d7-89d5-92941a8cebc9 None None] MFA Rules not processed for user `1bfb479e9e1e482e9c4bebda51edfe65`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:37.368383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-475310d1-058f-40d7-89d5-92941a8cebc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:37.394875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:37.447887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0620b74-58bf-411b-aac2-8ff3edc9821b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.448628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0620b74-58bf-411b-aac2-8ff3edc9821b admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.449670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0620b74-58bf-411b-aac2-8ff3edc9821b admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.449820 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0620b74-58bf-411b-aac2-8ff3edc9821b admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.450481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0620b74-58bf-411b-aac2-8ff3edc9821b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:37.452401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0620b74-58bf-411b-aac2-8ff3edc9821b admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:37.461071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 934/1878] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:37.461946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475310d1-058f-40d7-89d5-92941a8cebc9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.463221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-475310d1-058f-40d7-89d5-92941a8cebc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:37.481853 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475310d1-058f-40d7-89d5-92941a8cebc9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.487548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.488439 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.488799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.489088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] PATH_INFO: `/v3/users/561ef03eecbf447f9a5c94f3f0851c75` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.490404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=561ef03eecbf447f9a5c94f3f0851c75)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:37.491548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 945/1879] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:37 2022] POST /identity/v3/projects => generated 367 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:37.492389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0620b74-58bf-411b-aac2-8ff3edc9821b admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.492722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:37.495196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f0620b74-58bf-411b-aac2-8ff3edc9821b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:37.587454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '561ef03eecbf447f9a5c94f3f0851c75'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:37.606549 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '561ef03eecbf447f9a5c94f3f0851c75'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:37.628068 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:37.636104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '561ef03eecbf447f9a5c94f3f0851c75'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:37.648617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0620b74-58bf-411b-aac2-8ff3edc9821b admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.651180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '561ef03eecbf447f9a5c94f3f0851c75'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:37.652765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '561ef03eecbf447f9a5c94f3f0851c75'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:37.658460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c89f44-f1c6-4457-b3cb-3e774f19f266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.659201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c89f44-f1c6-4457-b3cb-3e774f19f266 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.659595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c89f44-f1c6-4457-b3cb-3e774f19f266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.659824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c89f44-f1c6-4457-b3cb-3e774f19f266 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.660698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c89f44-f1c6-4457-b3cb-3e774f19f266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:37.663036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c89f44-f1c6-4457-b3cb-3e774f19f266 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:37.670535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 935/1880] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:37 2022] DELETE /identity/v3/users/561ef03eecbf447f9a5c94f3f0851c75 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:37.671548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.674227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:37.720582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 946/1881] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:37 2022] POST /identity/v3/projects => generated 387 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:37.722897 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374128b-9e95-4035-ba11-a35d7ebcc339 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.723196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374128b-9e95-4035-ba11-a35d7ebcc339 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.723485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374128b-9e95-4035-ba11-a35d7ebcc339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.745418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-b374128b-9e95-4035-ba11-a35d7ebcc339 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:37.758454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b374128b-9e95-4035-ba11-a35d7ebcc339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:37.852185 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d4be1-828b-4008-9cef-ac2770a1ce11 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.952072 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 947/1882] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:37.953969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374128b-9e95-4035-ba11-a35d7ebcc339 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.955776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b374128b-9e95-4035-ba11-a35d7ebcc339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:37.987267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374128b-9e95-4035-ba11-a35d7ebcc339 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.009381 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64974d-f031-4d42-9d67-e328dc1acb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.010566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64974d-f031-4d42-9d67-e328dc1acb6c admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.010955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64974d-f031-4d42-9d67-e328dc1acb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.011409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64974d-f031-4d42-9d67-e328dc1acb6c admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.012467 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64974d-f031-4d42-9d67-e328dc1acb6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.015034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f64974d-f031-4d42-9d67-e328dc1acb6c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.072951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 948/1883] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:37 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:38.075103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64974d-f031-4d42-9d67-e328dc1acb6c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.077459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf1b61-eee6-469d-b3df-f76b8fec6ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.078158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8f64974d-f031-4d42-9d67-e328dc1acb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:38.078350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf1b61-eee6-469d-b3df-f76b8fec6ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.078820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf1b61-eee6-469d-b3df-f76b8fec6ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.079368 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf1b61-eee6-469d-b3df-f76b8fec6ca3 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.080249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf1b61-eee6-469d-b3df-f76b8fec6ca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.083429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf1b61-eee6-469d-b3df-f76b8fec6ca3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.142837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 936/1884] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:37 2022] POST /identity/v3/users => generated 284 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:38.142837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf1b61-eee6-469d-b3df-f76b8fec6ca3 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.157323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:38.195586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf1b61-eee6-469d-b3df-f76b8fec6ca3 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.212622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64974d-f031-4d42-9d67-e328dc1acb6c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.217674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b223756-6a70-4862-82a4-78c0452650f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.218721 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b223756-6a70-4862-82a4-78c0452650f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.219163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b223756-6a70-4862-82a4-78c0452650f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.219594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b223756-6a70-4862-82a4-78c0452650f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.223083 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e07058-91e5-4138-bdc0-c4580f66d302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.224315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e07058-91e5-4138-bdc0-c4580f66d302 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.224857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e07058-91e5-4138-bdc0-c4580f66d302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.225989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e07058-91e5-4138-bdc0-c4580f66d302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.237324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b223756-6a70-4862-82a4-78c0452650f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.237785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e07058-91e5-4138-bdc0-c4580f66d302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.238795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b223756-6a70-4862-82a4-78c0452650f9 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.239939 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e07058-91e5-4138-bdc0-c4580f66d302 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.275058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 949/1885] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:38.278178 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b223756-6a70-4862-82a4-78c0452650f9 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.304408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b223756-6a70-4862-82a4-78c0452650f9 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.328252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b944-8ab9-40d0-8c74-e4408cb721fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.329583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b944-8ab9-40d0-8c74-e4408cb721fc admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.330123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b944-8ab9-40d0-8c74-e4408cb721fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.330661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b944-8ab9-40d0-8c74-e4408cb721fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.331646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30b944-8ab9-40d0-8c74-e4408cb721fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.334342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30b944-8ab9-40d0-8c74-e4408cb721fc admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.338505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 937/1886] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:38.345924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e07058-91e5-4138-bdc0-c4580f66d302 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.374632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 950/1887] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:38.375780 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b944-8ab9-40d0-8c74-e4408cb721fc admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.391959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e07058-91e5-4138-bdc0-c4580f66d302 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.407146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bfc45-b5cb-4f8f-8785-d40ec9e1931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.407981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b944-8ab9-40d0-8c74-e4408cb721fc admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.410840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bfc45-b5cb-4f8f-8785-d40ec9e1931d admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.411574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bfc45-b5cb-4f8f-8785-d40ec9e1931d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.412162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bfc45-b5cb-4f8f-8785-d40ec9e1931d admin admin] PATH_INFO: `/v3/projects/a79b24f5e26e487fa667e0b328ae1709/users/31b6616a1d8141d285ca9f08d207d501/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.421931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a13ac-0c74-409b-b5d7-b4ee56549853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.423597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a13ac-0c74-409b-b5d7-b4ee56549853 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.423963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a13ac-0c74-409b-b5d7-b4ee56549853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.425876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a13ac-0c74-409b-b5d7-b4ee56549853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.426885 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a13ac-0c74-409b-b5d7-b4ee56549853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.428993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a13ac-0c74-409b-b5d7-b4ee56549853 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.468283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 951/1888] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:38.473682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a13ac-0c74-409b-b5d7-b4ee56549853 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.494384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a13ac-0c74-409b-b5d7-b4ee56549853 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.495323 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1bfc45-b5cb-4f8f-8785-d40ec9e1931d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a79b24f5e26e487fa667e0b328ae1709, user_id=31b6616a1d8141d285ca9f08d207d501, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.504098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1bfc45-b5cb-4f8f-8785-d40ec9e1931d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.505024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218da24-070b-495f-bb5f-4a2a74dcb91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.506346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218da24-070b-495f-bb5f-4a2a74dcb91e admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.506638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218da24-070b-495f-bb5f-4a2a74dcb91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.507169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218da24-070b-495f-bb5f-4a2a74dcb91e admin admin] PATH_INFO: `/v3/projects/0f19cedc2d4f41379acdb57fcb4318ef/users/9f4b4d6ce80c464199a49ed58f8a9b2c/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.538712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218da24-070b-495f-bb5f-4a2a74dcb91e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f19cedc2d4f41379acdb57fcb4318ef, user_id=9f4b4d6ce80c464199a49ed58f8a9b2c, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.545275 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218da24-070b-495f-bb5f-4a2a74dcb91e admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.545621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 938/1889] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:38 2022] PUT /identity/v3/projects/a79b24f5e26e487fa667e0b328ae1709/users/31b6616a1d8141d285ca9f08d207d501/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:38.550271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bfc45-b5cb-4f8f-8785-d40ec9e1931d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.603082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 952/1890] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:38 2022] PUT /identity/v3/projects/0f19cedc2d4f41379acdb57fcb4318ef/users/9f4b4d6ce80c464199a49ed58f8a9b2c/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:38.614849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7430120-9ee9-4e0c-a3aa-980e15bfa03e None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.615111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7430120-9ee9-4e0c-a3aa-980e15bfa03e None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.615495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7430120-9ee9-4e0c-a3aa-980e15bfa03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.727307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-d7430120-9ee9-4e0c-a3aa-980e15bfa03e None None] MFA Rules not processed for user `9f4b4d6ce80c464199a49ed58f8a9b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:38.728601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d7430120-9ee9-4e0c-a3aa-980e15bfa03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:38.798897 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:38.853135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 953/1891] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:38.854484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7430120-9ee9-4e0c-a3aa-980e15bfa03e None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.913176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bfc45-b5cb-4f8f-8785-d40ec9e1931d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.022244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7430120-9ee9-4e0c-a3aa-980e15bfa03e None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.030235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ffe4c-91d0-47d1-8e75-245b73ca8694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.031392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ffe4c-91d0-47d1-8e75-245b73ca8694 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.031886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ffe4c-91d0-47d1-8e75-245b73ca8694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.033032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ffe4c-91d0-47d1-8e75-245b73ca8694 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.033032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ffe4c-91d0-47d1-8e75-245b73ca8694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.035506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ffe4c-91d0-47d1-8e75-245b73ca8694 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.065500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:39.066819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c9374-8176-438f-9d8d-278b81e4265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.068635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c9374-8176-438f-9d8d-278b81e4265c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.069454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c9374-8176-438f-9d8d-278b81e4265c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.070150 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c9374-8176-438f-9d8d-278b81e4265c admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.072301 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c9374-8176-438f-9d8d-278b81e4265c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.077457 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c9374-8176-438f-9d8d-278b81e4265c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.093209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 954/1892] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:38 2022] POST /identity/v3/projects => generated 365 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:39.099124 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ffe4c-91d0-47d1-8e75-245b73ca8694 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.106237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 939/1893] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 556 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:39.111135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c9374-8176-438f-9d8d-278b81e4265c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.139835 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c9374-8176-438f-9d8d-278b81e4265c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.232795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ffe4c-91d0-47d1-8e75-245b73ca8694 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.239025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40851323-fa91-4a8a-a63b-40ed73336fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.239747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40851323-fa91-4a8a-a63b-40ed73336fac admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.240043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40851323-fa91-4a8a-a63b-40ed73336fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.240324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40851323-fa91-4a8a-a63b-40ed73336fac admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.241159 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40851323-fa91-4a8a-a63b-40ed73336fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.243345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40851323-fa91-4a8a-a63b-40ed73336fac admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.274848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 955/1894] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:39 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:39.280591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40851323-fa91-4a8a-a63b-40ed73336fac admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.280591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-40851323-fa91-4a8a-a63b-40ed73336fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:39.291936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c32d3-5641-4d30-8c04-2710f2b5f8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.293422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40851323-fa91-4a8a-a63b-40ed73336fac admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.296272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c32d3-5641-4d30-8c04-2710f2b5f8be admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.297093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c32d3-5641-4d30-8c04-2710f2b5f8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.298044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c32d3-5641-4d30-8c04-2710f2b5f8be admin admin] PATH_INFO: `/v3/projects/a79b24f5e26e487fa667e0b328ae1709/users/31b6616a1d8141d285ca9f08d207d501/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.298876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8716530-699b-4183-a0a9-3abfc7d16f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.299877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8716530-699b-4183-a0a9-3abfc7d16f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.300277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8716530-699b-4183-a0a9-3abfc7d16f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.300693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8716530-699b-4183-a0a9-3abfc7d16f49 admin admin] PATH_INFO: `/v3/projects/8ced3c5aa3994189a43d5eb81dd9d657` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.302328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8716530-699b-4183-a0a9-3abfc7d16f49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ced3c5aa3994189a43d5eb81dd9d657)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.304555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8716530-699b-4183-a0a9-3abfc7d16f49 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.305269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c32d3-5641-4d30-8c04-2710f2b5f8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=a79b24f5e26e487fa667e0b328ae1709, user_id=31b6616a1d8141d285ca9f08d207d501, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.311722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c32d3-5641-4d30-8c04-2710f2b5f8be admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.343234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 940/1895] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:39 2022] PUT /identity/v3/projects/a79b24f5e26e487fa667e0b328ae1709/users/31b6616a1d8141d285ca9f08d207d501/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:39.346098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c32d3-5641-4d30-8c04-2710f2b5f8be admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.383704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f8716530-699b-4183-a0a9-3abfc7d16f49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ced3c5aa3994189a43d5eb81dd9d657', 'request_id': 'req-f8716530-699b-4183-a0a9-3abfc7d16f49'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:39.401238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f8716530-699b-4183-a0a9-3abfc7d16f49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ced3c5aa3994189a43d5eb81dd9d657', 'request_id': 'req-f8716530-699b-4183-a0a9-3abfc7d16f49'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:39.420508 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f8716530-699b-4183-a0a9-3abfc7d16f49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ced3c5aa3994189a43d5eb81dd9d657', 'request_id': 'req-f8716530-699b-4183-a0a9-3abfc7d16f49'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:39.436209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f8716530-699b-4183-a0a9-3abfc7d16f49 admin admin] The token cache is being invalidate because project 8ced3c5aa3994189a43d5eb81dd9d657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:39.436691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f8716530-699b-4183-a0a9-3abfc7d16f49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:39.437513 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f8716530-699b-4183-a0a9-3abfc7d16f49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:39.444239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 956/1896] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:39 2022] DELETE /identity/v3/projects/8ced3c5aa3994189a43d5eb81dd9d657 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:39.444239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768958e8-e01b-499f-b95d-59b537f28bc1 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.444239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768958e8-e01b-499f-b95d-59b537f28bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.444239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768958e8-e01b-499f-b95d-59b537f28bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.495414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-768958e8-e01b-499f-b95d-59b537f28bc1 None None] MFA Rules not processed for user `31b6616a1d8141d285ca9f08d207d501`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:39.496747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-768958e8-e01b-499f-b95d-59b537f28bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:39.497906 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c32d3-5641-4d30-8c04-2710f2b5f8be admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.549343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:39.584540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:39.589888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e39bd-dd85-4311-8eb6-7d4aa61488b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.590690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e39bd-dd85-4311-8eb6-7d4aa61488b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.591049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e39bd-dd85-4311-8eb6-7d4aa61488b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.591442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e39bd-dd85-4311-8eb6-7d4aa61488b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.592336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e39bd-dd85-4311-8eb6-7d4aa61488b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.594597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e39bd-dd85-4311-8eb6-7d4aa61488b5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.602069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 957/1897] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:39.603264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768958e8-e01b-499f-b95d-59b537f28bc1 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.605845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-768958e8-e01b-499f-b95d-59b537f28bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:39.622269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 941/1898] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:39.623366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e39bd-dd85-4311-8eb6-7d4aa61488b5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.626725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-293e39bd-dd85-4311-8eb6-7d4aa61488b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:39.641784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768958e8-e01b-499f-b95d-59b537f28bc1 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.654209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.655996 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.656822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.657398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] PATH_INFO: `/v3/users/2daef1434fff40b8b5732294a3a26258` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.660510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2daef1434fff40b8b5732294a3a26258)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.664346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.735859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e39bd-dd85-4311-8eb6-7d4aa61488b5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.739662 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a200a41-ab19-49f1-8e6d-a73302f21994 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.740422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a200a41-ab19-49f1-8e6d-a73302f21994 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.740624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a200a41-ab19-49f1-8e6d-a73302f21994 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.740910 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a200a41-ab19-49f1-8e6d-a73302f21994 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.748094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2daef1434fff40b8b5732294a3a26258'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:39.764471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a200a41-ab19-49f1-8e6d-a73302f21994 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.765916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a200a41-ab19-49f1-8e6d-a73302f21994 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.766136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2daef1434fff40b8b5732294a3a26258'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:39.767094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2daef1434fff40b8b5732294a3a26258'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:39.779835 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2daef1434fff40b8b5732294a3a26258'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:39.779835 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2daef1434fff40b8b5732294a3a26258'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:39.792179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 942/1899] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:39.793759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a200a41-ab19-49f1-8e6d-a73302f21994 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.801915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9a200a41-ab19-49f1-8e6d-a73302f21994 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:39.806298 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 958/1900] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:39 2022] DELETE /identity/v3/users/2daef1434fff40b8b5732294a3a26258 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:39.863722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a200a41-ab19-49f1-8e6d-a73302f21994 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:40.001828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617952fb-379f-4483-bdba-9e9628c90a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:40.004030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617952fb-379f-4483-bdba-9e9628c90a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:40.004654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617952fb-379f-4483-bdba-9e9628c90a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:40.005541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617952fb-379f-4483-bdba-9e9628c90a7b admin admin] PATH_INFO: `/v3/projects/6b923ba16a2d49ea91f5c28b56941045/users/fde5c764ade14ae28d8cd49a4100f2a2/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.050197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617952fb-379f-4483-bdba-9e9628c90a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b923ba16a2d49ea91f5c28b56941045, user_id=fde5c764ade14ae28d8cd49a4100f2a2, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:40.053912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617952fb-379f-4483-bdba-9e9628c90a7b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:40.096442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 943/1901] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:39 2022] PUT /identity/v3/projects/6b923ba16a2d49ea91f5c28b56941045/users/fde5c764ade14ae28d8cd49a4100f2a2/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:40.111307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:40.119209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:40.243918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4d0a3-e7c2-41ff-8180-6b7189922ee1 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:40.518957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e46d0e-5e35-4156-84ca-be9d6e8c2490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:40.519303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e46d0e-5e35-4156-84ca-be9d6e8c2490 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:40.519593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e46d0e-5e35-4156-84ca-be9d6e8c2490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:40.519947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e46d0e-5e35-4156-84ca-be9d6e8c2490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.527667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e46d0e-5e35-4156-84ca-be9d6e8c2490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:40.529687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e46d0e-5e35-4156-84ca-be9d6e8c2490 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:40.544463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 959/1902] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:40.551269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617952fb-379f-4483-bdba-9e9628c90a7b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:40.579168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617952fb-379f-4483-bdba-9e9628c90a7b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:40.602610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8302b6-77c5-45c7-a0cd-80a0972deb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:40.603634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8302b6-77c5-45c7-a0cd-80a0972deb28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:40.604053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8302b6-77c5-45c7-a0cd-80a0972deb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:40.604435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8302b6-77c5-45c7-a0cd-80a0972deb28 admin admin] PATH_INFO: `/v3/projects/6b923ba16a2d49ea91f5c28b56941045/users/fde5c764ade14ae28d8cd49a4100f2a2/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.612626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8302b6-77c5-45c7-a0cd-80a0972deb28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b923ba16a2d49ea91f5c28b56941045, user_id=fde5c764ade14ae28d8cd49a4100f2a2, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:40.615581 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8302b6-77c5-45c7-a0cd-80a0972deb28 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:40.668517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 944/1903] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:40 2022] PUT /identity/v3/projects/6b923ba16a2d49ea91f5c28b56941045/users/fde5c764ade14ae28d8cd49a4100f2a2/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:40.678495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95637988-4624-4bbc-ac88-605f4db9b82f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:40.679709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95637988-4624-4bbc-ac88-605f4db9b82f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:40.681941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95637988-4624-4bbc-ac88-605f4db9b82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.837957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-95637988-4624-4bbc-ac88-605f4db9b82f None None] MFA Rules not processed for user `fde5c764ade14ae28d8cd49a4100f2a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:40.840431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-95637988-4624-4bbc-ac88-605f4db9b82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:40.849360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8302b6-77c5-45c7-a0cd-80a0972deb28 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:40.852104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fe8302b6-77c5-45c7-a0cd-80a0972deb28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.000446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:41.044590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:41.076939 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8302b6-77c5-45c7-a0cd-80a0972deb28 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:41.094300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd8112-771f-43b0-b103-d657e773f00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:41.096858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd8112-771f-43b0-b103-d657e773f00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.097675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd8112-771f-43b0-b103-d657e773f00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.098765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd8112-771f-43b0-b103-d657e773f00e admin admin] PATH_INFO: `/v3/projects/2a2657bc1d3c44b1b73af9273d80e032` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.099303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 960/1904] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:41.102629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd8112-771f-43b0-b103-d657e773f00e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a2657bc1d3c44b1b73af9273d80e032)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:41.108649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd8112-771f-43b0-b103-d657e773f00e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:41.175803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d34d6-cffe-4f35-a90c-9c1838725f76 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.176437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d34d6-cffe-4f35-a90c-9c1838725f76 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.176869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d34d6-cffe-4f35-a90c-9c1838725f76 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.180357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 961/1905] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 22:34:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:41.244107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d34d6-cffe-4f35-a90c-9c1838725f76 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:41.247214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b35d34d6-cffe-4f35-a90c-9c1838725f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.267258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bffd8112-771f-43b0-b103-d657e773f00e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a2657bc1d3c44b1b73af9273d80e032', 'request_id': 'req-bffd8112-771f-43b0-b103-d657e773f00e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:41.290902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bffd8112-771f-43b0-b103-d657e773f00e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a2657bc1d3c44b1b73af9273d80e032', 'request_id': 'req-bffd8112-771f-43b0-b103-d657e773f00e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:41.305248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bffd8112-771f-43b0-b103-d657e773f00e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a2657bc1d3c44b1b73af9273d80e032', 'request_id': 'req-bffd8112-771f-43b0-b103-d657e773f00e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:41.344202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bffd8112-771f-43b0-b103-d657e773f00e admin admin] The token cache is being invalidate because project 2a2657bc1d3c44b1b73af9273d80e032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:41.346163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bffd8112-771f-43b0-b103-d657e773f00e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:41.349325 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bffd8112-771f-43b0-b103-d657e773f00e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:41.353319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 945/1906] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:40 2022] DELETE /identity/v3/projects/2a2657bc1d3c44b1b73af9273d80e032 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:41.361102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d15161-28f5-4f37-8b8a-dc083544fc22 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.361695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d15161-28f5-4f37-8b8a-dc083544fc22 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.362831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d15161-28f5-4f37-8b8a-dc083544fc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.415567 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c5d15161-28f5-4f37-8b8a-dc083544fc22 None None] MFA Rules not processed for user `c77f6ccc7cb84c04ac532007a04a0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:41.419002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c5d15161-28f5-4f37-8b8a-dc083544fc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.443075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d34d6-cffe-4f35-a90c-9c1838725f76 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:41.455956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69246fa5-15c7-4c2d-ae00-9e3d26dac8fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:41.457104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69246fa5-15c7-4c2d-ae00-9e3d26dac8fd service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.457659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69246fa5-15c7-4c2d-ae00-9e3d26dac8fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.458037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69246fa5-15c7-4c2d-ae00-9e3d26dac8fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.463971 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-69246fa5-15c7-4c2d-ae00-9e3d26dac8fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.588308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 946/1907] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 22:34:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:41.595195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303ff2-7320-4b41-bd55-a3a3d37e0cf4 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.595717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303ff2-7320-4b41-bd55-a3a3d37e0cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.596102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303ff2-7320-4b41-bd55-a3a3d37e0cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.602290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69246fa5-15c7-4c2d-ae00-9e3d26dac8fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:41.603984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69246fa5-15c7-4c2d-ae00-9e3d26dac8fd service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:41.625559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 962/1908] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:41.627012 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69246fa5-15c7-4c2d-ae00-9e3d26dac8fd service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:41.628719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-69246fa5-15c7-4c2d-ae00-9e3d26dac8fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.639614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c4303ff2-7320-4b41-bd55-a3a3d37e0cf4 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:41.642463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c4303ff2-7320-4b41-bd55-a3a3d37e0cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.651453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69246fa5-15c7-4c2d-ae00-9e3d26dac8fd service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:41.662349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd6d8c-fb0f-477c-838c-03a97c9a4f4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:41.664814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd6d8c-fb0f-477c-838c-03a97c9a4f4a service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.665388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd6d8c-fb0f-477c-838c-03a97c9a4f4a service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.665579 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd6d8c-fb0f-477c-838c-03a97c9a4f4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.694106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd6d8c-fb0f-477c-838c-03a97c9a4f4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:41.695727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd6d8c-fb0f-477c-838c-03a97c9a4f4a service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:41.723467 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 963/1909] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:41.790640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd6d8c-fb0f-477c-838c-03a97c9a4f4a service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:41.792273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bdcd6d8c-fb0f-477c-838c-03a97c9a4f4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.802066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 947/1910] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:41.810505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd6d8c-fb0f-477c-838c-03a97c9a4f4a service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:41.814969 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966d036-51c7-4535-bb75-0cf1c359903d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.815338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966d036-51c7-4535-bb75-0cf1c359903d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.815681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966d036-51c7-4535-bb75-0cf1c359903d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.815975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1bfab-517f-4cd3-8739-93f37431b942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:41.816639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1bfab-517f-4cd3-8739-93f37431b942 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.816939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1bfab-517f-4cd3-8739-93f37431b942 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.817208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1bfab-517f-4cd3-8739-93f37431b942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.819144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-49a1bfab-517f-4cd3-8739-93f37431b942 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.825832 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:41.841276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5966d036-51c7-4535-bb75-0cf1c359903d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:41.843582 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5966d036-51c7-4535-bb75-0cf1c359903d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.852747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 948/1911] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:41.857682 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966d036-51c7-4535-bb75-0cf1c359903d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:41.859710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5966d036-51c7-4535-bb75-0cf1c359903d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.875401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966d036-51c7-4535-bb75-0cf1c359903d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:41.879700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfd1c8-5562-4a63-8ed2-5067d37dc686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:41.880350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfd1c8-5562-4a63-8ed2-5067d37dc686 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.880696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfd1c8-5562-4a63-8ed2-5067d37dc686 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.880978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfd1c8-5562-4a63-8ed2-5067d37dc686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.882845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-51bfd1c8-5562-4a63-8ed2-5067d37dc686 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.900501 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a1bfab-517f-4cd3-8739-93f37431b942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:41.902952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a1bfab-517f-4cd3-8739-93f37431b942 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:41.913030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 964/1912] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:41 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:41.913978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1bfab-517f-4cd3-8739-93f37431b942 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:41.926542 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1bfab-517f-4cd3-8739-93f37431b942 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:41.931810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23614f-1f2a-4e49-918e-412c961de339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:41.932562 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23614f-1f2a-4e49-918e-412c961de339 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.933449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23614f-1f2a-4e49-918e-412c961de339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.933749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23614f-1f2a-4e49-918e-412c961de339 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.935607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23614f-1f2a-4e49-918e-412c961de339 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:41.935607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23614f-1f2a-4e49-918e-412c961de339 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:41.937393 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23614f-1f2a-4e49-918e-412c961de339 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:41.956690 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 965/1913] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:41.958512 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23614f-1f2a-4e49-918e-412c961de339 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:41.970269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23614f-1f2a-4e49-918e-412c961de339 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:41.978120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723854b-fb49-4b27-a444-28df3406ce50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:41.978120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723854b-fb49-4b27-a444-28df3406ce50 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.978569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723854b-fb49-4b27-a444-28df3406ce50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.978668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723854b-fb49-4b27-a444-28df3406ce50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.979670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723854b-fb49-4b27-a444-28df3406ce50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:41.982555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723854b-fb49-4b27-a444-28df3406ce50 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:41.983716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfd1c8-5562-4a63-8ed2-5067d37dc686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:41.988825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:41.998838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfd1c8-5562-4a63-8ed2-5067d37dc686 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:42.021328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 949/1914] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:42.037246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 966/1915] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:41 2022] POST /identity/v3/projects => generated 407 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:42.041655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfd1c8-5562-4a63-8ed2-5067d37dc686 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.063779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfd1c8-5562-4a63-8ed2-5067d37dc686 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:42.181978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f2551-e9fc-45fc-bad4-c67d82dc96ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.184041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f2551-e9fc-45fc-bad4-c67d82dc96ce admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.184764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f2551-e9fc-45fc-bad4-c67d82dc96ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.185269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f2551-e9fc-45fc-bad4-c67d82dc96ce admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.193348 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f2551-e9fc-45fc-bad4-c67d82dc96ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:42.195886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f2551-e9fc-45fc-bad4-c67d82dc96ce admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:42.233337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 950/1916] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:42 2022] POST /identity/v3/users => generated 304 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:42.237844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723854b-fb49-4b27-a444-28df3406ce50 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.259653 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723854b-fb49-4b27-a444-28df3406ce50 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:42.289191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecab0c9-3be7-43cf-b38e-82812a6af683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.290067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecab0c9-3be7-43cf-b38e-82812a6af683 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.290534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecab0c9-3be7-43cf-b38e-82812a6af683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.290955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecab0c9-3be7-43cf-b38e-82812a6af683 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.291739 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecab0c9-3be7-43cf-b38e-82812a6af683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:42.294831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecab0c9-3be7-43cf-b38e-82812a6af683 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:42.312524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 967/1917] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:42.315867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f2551-e9fc-45fc-bad4-c67d82dc96ce admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.345828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f2551-e9fc-45fc-bad4-c67d82dc96ce admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:42.353071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2aadb1-83f2-40a9-8fa3-88d576f90aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.354035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aadb1-83f2-40a9-8fa3-88d576f90aba admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.354438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aadb1-83f2-40a9-8fa3-88d576f90aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.355025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aadb1-83f2-40a9-8fa3-88d576f90aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.357380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2aadb1-83f2-40a9-8fa3-88d576f90aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:42.359355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2aadb1-83f2-40a9-8fa3-88d576f90aba admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:42.389752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 951/1918] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:42.392441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecab0c9-3be7-43cf-b38e-82812a6af683 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.414807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecab0c9-3be7-43cf-b38e-82812a6af683 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:42.446517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe5419-3c93-4927-b61a-1a964e734f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.449358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe5419-3c93-4927-b61a-1a964e734f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.450042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe5419-3c93-4927-b61a-1a964e734f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.451441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe5419-3c93-4927-b61a-1a964e734f3c admin admin] PATH_INFO: `/v3/projects/61cede8d05234ff09c4f13b95d8189b7/users/8ffcf6bceb0f4d1c93745dac18b3bf8c/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.498879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe5419-3c93-4927-b61a-1a964e734f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=61cede8d05234ff09c4f13b95d8189b7, user_id=8ffcf6bceb0f4d1c93745dac18b3bf8c, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:42.505628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe5419-3c93-4927-b61a-1a964e734f3c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:42.567609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 968/1919] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:42 2022] PUT /identity/v3/projects/61cede8d05234ff09c4f13b95d8189b7/users/8ffcf6bceb0f4d1c93745dac18b3bf8c/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:42.576285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cb09f-33ca-417e-9eb2-5fff0f51acf6 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.576911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cb09f-33ca-417e-9eb2-5fff0f51acf6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.577233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cb09f-33ca-417e-9eb2-5fff0f51acf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.643930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-518cb09f-33ca-417e-9eb2-5fff0f51acf6 None None] MFA Rules not processed for user `8ffcf6bceb0f4d1c93745dac18b3bf8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:42.650442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-518cb09f-33ca-417e-9eb2-5fff0f51acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:42.810288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 952/1920] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:42 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:42.816108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe5419-3c93-4927-b61a-1a964e734f3c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.976359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe5419-3c93-4927-b61a-1a964e734f3c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:42.993925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037ca64-b95c-4b85-92dc-0905d8c74b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.995211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037ca64-b95c-4b85-92dc-0905d8c74b5f admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.996061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037ca64-b95c-4b85-92dc-0905d8c74b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.996351 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037ca64-b95c-4b85-92dc-0905d8c74b5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.997131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037ca64-b95c-4b85-92dc-0905d8c74b5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:42.999138 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037ca64-b95c-4b85-92dc-0905d8c74b5f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.080494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 969/1921] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:42 2022] POST /identity/v3/projects => generated 407 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:43.088452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cb09f-33ca-417e-9eb2-5fff0f51acf6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.096920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:43.103767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cb09f-33ca-417e-9eb2-5fff0f51acf6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.187498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da6159-fc64-4434-b0d5-875a01a9afa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.188238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da6159-fc64-4434-b0d5-875a01a9afa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.188754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da6159-fc64-4434-b0d5-875a01a9afa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.189064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da6159-fc64-4434-b0d5-875a01a9afa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.189947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da6159-fc64-4434-b0d5-875a01a9afa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.192546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da6159-fc64-4434-b0d5-875a01a9afa4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.248558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 953/1922] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:43 2022] POST /identity/v3/users => generated 310 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:43.254427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037ca64-b95c-4b85-92dc-0905d8c74b5f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.296738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037ca64-b95c-4b85-92dc-0905d8c74b5f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.318024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e15f1-eaf0-4096-9ee5-e368fb67e43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.319450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e15f1-eaf0-4096-9ee5-e368fb67e43e admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.320064 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e15f1-eaf0-4096-9ee5-e368fb67e43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.320674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e15f1-eaf0-4096-9ee5-e368fb67e43e admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.322104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e15f1-eaf0-4096-9ee5-e368fb67e43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.330595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e15f1-eaf0-4096-9ee5-e368fb67e43e admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.351466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 970/1923] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:43.357173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da6159-fc64-4434-b0d5-875a01a9afa4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.372779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da6159-fc64-4434-b0d5-875a01a9afa4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.389990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf5d80-d5c8-4695-aa2b-ea38031e0eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.391317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf5d80-d5c8-4695-aa2b-ea38031e0eb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.393332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf5d80-d5c8-4695-aa2b-ea38031e0eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.393622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf5d80-d5c8-4695-aa2b-ea38031e0eb4 admin admin] PATH_INFO: `/v3/projects/76d27dab0d14438a8714eb2e94257630/users/235f70bfd4d04298aa1604483dbc7344/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.432266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf5d80-d5c8-4695-aa2b-ea38031e0eb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d27dab0d14438a8714eb2e94257630, user_id=235f70bfd4d04298aa1604483dbc7344, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.436268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf5d80-d5c8-4695-aa2b-ea38031e0eb4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.469257 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 954/1924] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:43 2022] PUT /identity/v3/projects/76d27dab0d14438a8714eb2e94257630/users/235f70bfd4d04298aa1604483dbc7344/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:43.473111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e15f1-eaf0-4096-9ee5-e368fb67e43e admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.486986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e15f1-eaf0-4096-9ee5-e368fb67e43e admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.575983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b99d4-f5af-4d7e-8be3-9369944f0eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.577085 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b99d4-f5af-4d7e-8be3-9369944f0eaa admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.577552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b99d4-f5af-4d7e-8be3-9369944f0eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.578016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b99d4-f5af-4d7e-8be3-9369944f0eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.579272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b99d4-f5af-4d7e-8be3-9369944f0eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.582412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b99d4-f5af-4d7e-8be3-9369944f0eaa admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.604552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 971/1925] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:43.606122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf5d80-d5c8-4695-aa2b-ea38031e0eb4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.622756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf5d80-d5c8-4695-aa2b-ea38031e0eb4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.627851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a46f5-7ec6-4806-b959-ed30d5da0259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.628859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a46f5-7ec6-4806-b959-ed30d5da0259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.629142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a46f5-7ec6-4806-b959-ed30d5da0259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.629421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a46f5-7ec6-4806-b959-ed30d5da0259 admin admin] PATH_INFO: `/v3/projects/76d27dab0d14438a8714eb2e94257630/users/235f70bfd4d04298aa1604483dbc7344/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.631474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a46f5-7ec6-4806-b959-ed30d5da0259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d27dab0d14438a8714eb2e94257630, user_id=235f70bfd4d04298aa1604483dbc7344, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.634512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a46f5-7ec6-4806-b959-ed30d5da0259 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.656337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 955/1926] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:43 2022] PUT /identity/v3/projects/76d27dab0d14438a8714eb2e94257630/users/235f70bfd4d04298aa1604483dbc7344/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:43.670114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e8797-e945-41dd-a288-f3f3dfd0da13 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.670698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e8797-e945-41dd-a288-f3f3dfd0da13 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.671303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e8797-e945-41dd-a288-f3f3dfd0da13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.715286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-497e8797-e945-41dd-a288-f3f3dfd0da13 None None] MFA Rules not processed for user `235f70bfd4d04298aa1604483dbc7344`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:43.716734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-497e8797-e945-41dd-a288-f3f3dfd0da13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:43.728639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:43.823920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 972/1927] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:43 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:43.838927 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a46f5-7ec6-4806-b959-ed30d5da0259 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.945576 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a46f5-7ec6-4806-b959-ed30d5da0259 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.949374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d399181-ec87-494d-bd7e-78dc495f4b78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.950043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d399181-ec87-494d-bd7e-78dc495f4b78 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.950326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d399181-ec87-494d-bd7e-78dc495f4b78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.950615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d399181-ec87-494d-bd7e-78dc495f4b78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.967092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d399181-ec87-494d-bd7e-78dc495f4b78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.970031 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d399181-ec87-494d-bd7e-78dc495f4b78 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.989888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 956/1928] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:44.367153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e8797-e945-41dd-a288-f3f3dfd0da13 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:44.379761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e8797-e945-41dd-a288-f3f3dfd0da13 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:44.385097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f8833-5e4a-4c4b-883f-167fc0f6afff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:44.386024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f8833-5e4a-4c4b-883f-167fc0f6afff service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:44.387127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f8833-5e4a-4c4b-883f-167fc0f6afff service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:44.387494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f8833-5e4a-4c4b-883f-167fc0f6afff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:44.514813 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f8833-5e4a-4c4b-883f-167fc0f6afff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:44.521580 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f8833-5e4a-4c4b-883f-167fc0f6afff service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:44.548026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 973/1929] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:44 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:47.351109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d399181-ec87-494d-bd7e-78dc495f4b78 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:47.361955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d399181-ec87-494d-bd7e-78dc495f4b78 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:47.367364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ab470-2c1a-4049-9f4f-3f71627bf2ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:47.368037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ab470-2c1a-4049-9f4f-3f71627bf2ee service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:47.368325 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ab470-2c1a-4049-9f4f-3f71627bf2ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:47.368676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ab470-2c1a-4049-9f4f-3f71627bf2ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:47.371842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fc3ab470-2c1a-4049-9f4f-3f71627bf2ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:47.452942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ab470-2c1a-4049-9f4f-3f71627bf2ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:47.454514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ab470-2c1a-4049-9f4f-3f71627bf2ee service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:47.458769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:47.469167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 957/1930] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:34:48.119976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f8833-5e4a-4c4b-883f-167fc0f6afff service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:48.121896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-684f8833-5e4a-4c4b-883f-167fc0f6afff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:48.189929 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f8833-5e4a-4c4b-883f-167fc0f6afff service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:48.194445 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:48.195333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:48.195629 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:48.195874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] PATH_INFO: `/v3/users/9f4b4d6ce80c464199a49ed58f8a9b2c` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:48.197100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f4b4d6ce80c464199a49ed58f8a9b2c)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:48.199100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:48.246037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f4b4d6ce80c464199a49ed58f8a9b2c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:48.247675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:48.254306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f4b4d6ce80c464199a49ed58f8a9b2c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:48.255151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f4b4d6ce80c464199a49ed58f8a9b2c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:48.263219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f4b4d6ce80c464199a49ed58f8a9b2c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:48.263892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f4b4d6ce80c464199a49ed58f8a9b2c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:48.285555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 974/1931] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:48 2022] DELETE /identity/v3/users/9f4b4d6ce80c464199a49ed58f8a9b2c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:48.290411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68899bda-4ed7-4acf-bc2a-5f6a41697d18 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:48.290706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68899bda-4ed7-4acf-bc2a-5f6a41697d18 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:48.291006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68899bda-4ed7-4acf-bc2a-5f6a41697d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:48.311117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-68899bda-4ed7-4acf-bc2a-5f6a41697d18 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:48.312572 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-68899bda-4ed7-4acf-bc2a-5f6a41697d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:48.378107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 958/1932] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:48.389193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:48.390838 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:48.461171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0c135-fe29-4ab1-919d-bfd8d9abbb95 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:48.466310 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7079e-9d9a-430a-b92a-4245ba9a0cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:48.467212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7079e-9d9a-430a-b92a-4245ba9a0cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:48.468271 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7079e-9d9a-430a-b92a-4245ba9a0cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:48.468595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7079e-9d9a-430a-b92a-4245ba9a0cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:48.478456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7079e-9d9a-430a-b92a-4245ba9a0cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:48.481015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7079e-9d9a-430a-b92a-4245ba9a0cd8 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:48.491560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 975/1933] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:48.783116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68899bda-4ed7-4acf-bc2a-5f6a41697d18 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:48.784733 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-68899bda-4ed7-4acf-bc2a-5f6a41697d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:48.797641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68899bda-4ed7-4acf-bc2a-5f6a41697d18 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:48.802890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:48.803196 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:48.803484 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:48.803761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] PATH_INFO: `/v3/projects/0f19cedc2d4f41379acdb57fcb4318ef` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:48.805259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f19cedc2d4f41379acdb57fcb4318ef)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:48.807073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:48.868688 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f19cedc2d4f41379acdb57fcb4318ef', 'request_id': 'req-b7515df1-3922-4b65-be66-a6bd0a56216e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:48.874307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f19cedc2d4f41379acdb57fcb4318ef', 'request_id': 'req-b7515df1-3922-4b65-be66-a6bd0a56216e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:48.882012 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f19cedc2d4f41379acdb57fcb4318ef', 'request_id': 'req-b7515df1-3922-4b65-be66-a6bd0a56216e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:48.895262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] The token cache is being invalidate because project 0f19cedc2d4f41379acdb57fcb4318ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:48.895555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:48.896223 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:48.898029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 959/1934] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:48 2022] DELETE /identity/v3/projects/0f19cedc2d4f41379acdb57fcb4318ef => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:48.902457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7079e-9d9a-430a-b92a-4245ba9a0cd8 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:48.913109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7079e-9d9a-430a-b92a-4245ba9a0cd8 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:48.986716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ce410-21ff-493a-9d52-1f98e4fa1a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:48.988091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ce410-21ff-493a-9d52-1f98e4fa1a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:48.988459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ce410-21ff-493a-9d52-1f98e4fa1a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:48.988918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ce410-21ff-493a-9d52-1f98e4fa1a89 admin admin] PATH_INFO: `/v3/users/fde5c764ade14ae28d8cd49a4100f2a2` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:48.990419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ce410-21ff-493a-9d52-1f98e4fa1a89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde5c764ade14ae28d8cd49a4100f2a2)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:48.993337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ce410-21ff-493a-9d52-1f98e4fa1a89 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:49.050952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-c75ce410-21ff-493a-9d52-1f98e4fa1a89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde5c764ade14ae28d8cd49a4100f2a2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:49.059997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-c75ce410-21ff-493a-9d52-1f98e4fa1a89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde5c764ade14ae28d8cd49a4100f2a2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:49.060807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-c75ce410-21ff-493a-9d52-1f98e4fa1a89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde5c764ade14ae28d8cd49a4100f2a2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:49.069108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-c75ce410-21ff-493a-9d52-1f98e4fa1a89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde5c764ade14ae28d8cd49a4100f2a2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:49.069823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-c75ce410-21ff-493a-9d52-1f98e4fa1a89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde5c764ade14ae28d8cd49a4100f2a2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:49.075855 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:49.093384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 976/1935] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:48 2022] DELETE /identity/v3/users/fde5c764ade14ae28d8cd49a4100f2a2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:49.389458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:49.391076 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:49.406057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7515df1-3922-4b65-be66-a6bd0a56216e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:49.475637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e971f9-92e0-4125-9bc4-6a72555c01d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:49.476555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e971f9-92e0-4125-9bc4-6a72555c01d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.476980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e971f9-92e0-4125-9bc4-6a72555c01d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.477275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e971f9-92e0-4125-9bc4-6a72555c01d5 admin admin] PATH_INFO: `/v3/projects/6b923ba16a2d49ea91f5c28b56941045` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.483173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e971f9-92e0-4125-9bc4-6a72555c01d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b923ba16a2d49ea91f5c28b56941045)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:49.483173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e971f9-92e0-4125-9bc4-6a72555c01d5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:49.523387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:49.556363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e0e971f9-92e0-4125-9bc4-6a72555c01d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b923ba16a2d49ea91f5c28b56941045', 'request_id': 'req-e0e971f9-92e0-4125-9bc4-6a72555c01d5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:49.563894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e0e971f9-92e0-4125-9bc4-6a72555c01d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b923ba16a2d49ea91f5c28b56941045', 'request_id': 'req-e0e971f9-92e0-4125-9bc4-6a72555c01d5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:49.572988 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e0e971f9-92e0-4125-9bc4-6a72555c01d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b923ba16a2d49ea91f5c28b56941045', 'request_id': 'req-e0e971f9-92e0-4125-9bc4-6a72555c01d5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:49.586035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e0e971f9-92e0-4125-9bc4-6a72555c01d5 admin admin] The token cache is being invalidate because project 6b923ba16a2d49ea91f5c28b56941045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:49.586322 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e0e971f9-92e0-4125-9bc4-6a72555c01d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:49.587174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e0e971f9-92e0-4125-9bc4-6a72555c01d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:49.589846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 960/1936] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:49 2022] DELETE /identity/v3/projects/6b923ba16a2d49ea91f5c28b56941045 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:49.597401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e92a6-715a-4011-972e-6fb878bd994c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.598315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e92a6-715a-4011-972e-6fb878bd994c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.598611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e92a6-715a-4011-972e-6fb878bd994c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.619756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-8b6e92a6-715a-4011-972e-6fb878bd994c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:49.621441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8b6e92a6-715a-4011-972e-6fb878bd994c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:49.709257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 977/1937] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:49.720785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1790fb5-807c-492c-9400-075e09786c55 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.720785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1790fb5-807c-492c-9400-075e09786c55 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.720785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1790fb5-807c-492c-9400-075e09786c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.739747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e1790fb5-807c-492c-9400-075e09786c55 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:49.740995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e1790fb5-807c-492c-9400-075e09786c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:49.750059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 961/1938] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:49.754638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e92a6-715a-4011-972e-6fb878bd994c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:49.765941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e92a6-715a-4011-972e-6fb878bd994c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:49.770626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf913e0c-bafe-474e-8da3-f99b24b3804a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:49.771377 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf913e0c-bafe-474e-8da3-f99b24b3804a admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.771723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf913e0c-bafe-474e-8da3-f99b24b3804a admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.772011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf913e0c-bafe-474e-8da3-f99b24b3804a admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.772802 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf913e0c-bafe-474e-8da3-f99b24b3804a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:49.773113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf913e0c-bafe-474e-8da3-f99b24b3804a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:49.774762 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf913e0c-bafe-474e-8da3-f99b24b3804a admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:49.796600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 978/1939] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:49.798154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1790fb5-807c-492c-9400-075e09786c55 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:49.809920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1790fb5-807c-492c-9400-075e09786c55 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:49.819271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4d045-1d9e-4a9f-be96-8d29ff98aa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:49.820444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4d045-1d9e-4a9f-be96-8d29ff98aa1c admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.821010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4d045-1d9e-4a9f-be96-8d29ff98aa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.821547 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4d045-1d9e-4a9f-be96-8d29ff98aa1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.822898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4d045-1d9e-4a9f-be96-8d29ff98aa1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:49.827244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4d045-1d9e-4a9f-be96-8d29ff98aa1c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:49.878891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 962/1940] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:49 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:49.879190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf913e0c-bafe-474e-8da3-f99b24b3804a admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:49.890784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf913e0c-bafe-474e-8da3-f99b24b3804a admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:49.960373 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fa92a-e3d3-4ffc-9c5b-6f93fcdba214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:49.961272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fa92a-e3d3-4ffc-9c5b-6f93fcdba214 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.961720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fa92a-e3d3-4ffc-9c5b-6f93fcdba214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.962015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fa92a-e3d3-4ffc-9c5b-6f93fcdba214 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.962772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fa92a-e3d3-4ffc-9c5b-6f93fcdba214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:49.964831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fa92a-e3d3-4ffc-9c5b-6f93fcdba214 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:49.994497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 979/1941] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:49 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:50.005955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4d045-1d9e-4a9f-be96-8d29ff98aa1c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:50.017956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4d045-1d9e-4a9f-be96-8d29ff98aa1c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:50.023248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec242ba6-f31c-44d3-8c44-055aab91829f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:50.024250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec242ba6-f31c-44d3-8c44-055aab91829f admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.024799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec242ba6-f31c-44d3-8c44-055aab91829f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.025294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec242ba6-f31c-44d3-8c44-055aab91829f admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.026759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec242ba6-f31c-44d3-8c44-055aab91829f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:50.029986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec242ba6-f31c-44d3-8c44-055aab91829f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:50.051631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 963/1942] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:50.055930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fa92a-e3d3-4ffc-9c5b-6f93fcdba214 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:50.069315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fa92a-e3d3-4ffc-9c5b-6f93fcdba214 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:50.074678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4e8dc-ceb5-4b21-998e-561a37bb77b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:50.075370 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4e8dc-ceb5-4b21-998e-561a37bb77b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.075659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4e8dc-ceb5-4b21-998e-561a37bb77b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.075951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4e8dc-ceb5-4b21-998e-561a37bb77b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.076749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4e8dc-ceb5-4b21-998e-561a37bb77b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:50.078382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4e8dc-ceb5-4b21-998e-561a37bb77b7 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:50.090976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 980/1943] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:50.096535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec242ba6-f31c-44d3-8c44-055aab91829f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:50.110125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec242ba6-f31c-44d3-8c44-055aab91829f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:50.115540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc4120-76db-4862-9114-bf763749c7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:50.116441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc4120-76db-4862-9114-bf763749c7f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.116767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc4120-76db-4862-9114-bf763749c7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.117046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc4120-76db-4862-9114-bf763749c7f6 admin admin] PATH_INFO: `/v3/projects/ba02dca1ca1f40ec8e4b909eb3967d7f/users/7c6730fc49df438083b5da8f0c331f3f/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.133786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc4120-76db-4862-9114-bf763749c7f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba02dca1ca1f40ec8e4b909eb3967d7f, user_id=7c6730fc49df438083b5da8f0c331f3f, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:50.136937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc4120-76db-4862-9114-bf763749c7f6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:50.154393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 964/1944] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:50 2022] PUT /identity/v3/projects/ba02dca1ca1f40ec8e4b909eb3967d7f/users/7c6730fc49df438083b5da8f0c331f3f/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:50.164761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740b6cc-63cb-455a-92c7-fc369ed82325 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.165054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740b6cc-63cb-455a-92c7-fc369ed82325 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.165346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740b6cc-63cb-455a-92c7-fc369ed82325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.197741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-d740b6cc-63cb-455a-92c7-fc369ed82325 None None] MFA Rules not processed for user `7c6730fc49df438083b5da8f0c331f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:50.200074 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d740b6cc-63cb-455a-92c7-fc369ed82325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:50.270735 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:50.303843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 981/1945] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:50.319504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc4120-76db-4862-9114-bf763749c7f6 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:50.322046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-24cc4120-76db-4862-9114-bf763749c7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:50.447470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc4120-76db-4862-9114-bf763749c7f6 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:50.461658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed059bba-2972-4226-8529-7f460b50d365 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b67db080874f3ebb508239c7a8ccf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:50.462928 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed059bba-2972-4226-8529-7f460b50d365 service glance] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.463725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed059bba-2972-4226-8529-7f460b50d365 service glance] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.464915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed059bba-2972-4226-8529-7f460b50d365 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.478451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed059bba-2972-4226-8529-7f460b50d365 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:50.480362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed059bba-2972-4226-8529-7f460b50d365 service glance] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:50.491551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 965/1946] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:51.841336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740b6cc-63cb-455a-92c7-fc369ed82325 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:51.842938 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d740b6cc-63cb-455a-92c7-fc369ed82325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:51.959138 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740b6cc-63cb-455a-92c7-fc369ed82325 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:51.967047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:51.968831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:51.969007 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:51.969309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] PATH_INFO: `/v3/users/52ee3de3c8774fb99da017cda4a1e7aa` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:51.976537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ee3de3c8774fb99da017cda4a1e7aa)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:51.977666 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:52.105211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ee3de3c8774fb99da017cda4a1e7aa'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:52.126143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ee3de3c8774fb99da017cda4a1e7aa'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:52.127804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ee3de3c8774fb99da017cda4a1e7aa'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:52.153287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ee3de3c8774fb99da017cda4a1e7aa'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:52.154082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ee3de3c8774fb99da017cda4a1e7aa'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:52.188038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 982/1947] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:51 2022] DELETE /identity/v3/users/52ee3de3c8774fb99da017cda4a1e7aa => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:52.193022 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e93825-97ca-4a5f-b963-c7d6f49de078 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:52.193995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e93825-97ca-4a5f-b963-c7d6f49de078 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:52.193995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e93825-97ca-4a5f-b963-c7d6f49de078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:52.242272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-78e93825-97ca-4a5f-b963-c7d6f49de078 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:52.244132 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-78e93825-97ca-4a5f-b963-c7d6f49de078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:52.298826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:52.365429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 966/1948] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:34:52.387269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:52.389469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:52.504085 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857d090-4497-4930-8dbc-8c7617318532 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:52.508186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402ef7e-0445-45d5-9ddc-d2f03cc4072f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:52.508973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402ef7e-0445-45d5-9ddc-d2f03cc4072f service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:52.509258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402ef7e-0445-45d5-9ddc-d2f03cc4072f service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:52.509538 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402ef7e-0445-45d5-9ddc-d2f03cc4072f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:52.521844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402ef7e-0445-45d5-9ddc-d2f03cc4072f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:52.523664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402ef7e-0445-45d5-9ddc-d2f03cc4072f service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:52.537775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 983/1949] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:53.755703 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e93825-97ca-4a5f-b963-c7d6f49de078 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:53.757180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-78e93825-97ca-4a5f-b963-c7d6f49de078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:53.770604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e93825-97ca-4a5f-b963-c7d6f49de078 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:53.775358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:53.776930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.777269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.777548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] PATH_INFO: `/v3/projects/e488b9697cfd497e945cf37d6691548c` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.778850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e488b9697cfd497e945cf37d6691548c)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:53.780932 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:53.849769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e488b9697cfd497e945cf37d6691548c', 'request_id': 'req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:53.857760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e488b9697cfd497e945cf37d6691548c', 'request_id': 'req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:53.866231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e488b9697cfd497e945cf37d6691548c', 'request_id': 'req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:53.878156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] The token cache is being invalidate because project e488b9697cfd497e945cf37d6691548c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:53.878446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:53.879130 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:53.880723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 967/1950] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:53 2022] DELETE /identity/v3/projects/e488b9697cfd497e945cf37d6691548c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:53.886046 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402ef7e-0445-45d5-9ddc-d2f03cc4072f service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:53.892666 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:53.897097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402ef7e-0445-45d5-9ddc-d2f03cc4072f service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:53.961225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:53.961684 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.961967 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.962244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] PATH_INFO: `/v3/users/1bfb479e9e1e482e9c4bebda51edfe65` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.963441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bfb479e9e1e482e9c4bebda51edfe65)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:53.965490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:54.025418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bfb479e9e1e482e9c4bebda51edfe65'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:54.034613 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bfb479e9e1e482e9c4bebda51edfe65'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:54.035234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bfb479e9e1e482e9c4bebda51edfe65'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:54.043980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bfb479e9e1e482e9c4bebda51edfe65'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:54.044795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bfb479e9e1e482e9c4bebda51edfe65'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:54.064374 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 984/1951] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:53 2022] DELETE /identity/v3/users/1bfb479e9e1e482e9c4bebda51edfe65 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:54.435144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:54.438146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:54.462307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e1400-5ac1-4fd4-be3a-f0b696b54d69 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:54.614923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:54.614923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:54.614923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:54.614923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] PATH_INFO: `/v3/projects/1cb0b668382d4dcb8c7720178017c159` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:54.620938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cb0b668382d4dcb8c7720178017c159)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:54.623080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:54.746723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cb0b668382d4dcb8c7720178017c159', 'request_id': 'req-a8513dbb-5ae8-479f-b0bd-762b3f85270b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:54.753067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:54.755012 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cb0b668382d4dcb8c7720178017c159', 'request_id': 'req-a8513dbb-5ae8-479f-b0bd-762b3f85270b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:54.774039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cb0b668382d4dcb8c7720178017c159', 'request_id': 'req-a8513dbb-5ae8-479f-b0bd-762b3f85270b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:54.789908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] The token cache is being invalidate because project 1cb0b668382d4dcb8c7720178017c159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:54.790249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:54.791305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:54.793875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 968/1952] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:54 2022] DELETE /identity/v3/projects/1cb0b668382d4dcb8c7720178017c159 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:54.798642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:54.800325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:54.830279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09b2a0-8b64-42d6-9880-70f4d197c2af admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:54.947032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88142e35-bca0-4f35-9065-d6d59135a448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:54.948422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88142e35-bca0-4f35-9065-d6d59135a448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:54.948839 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88142e35-bca0-4f35-9065-d6d59135a448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:54.949275 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88142e35-bca0-4f35-9065-d6d59135a448 admin admin] PATH_INFO: `/v3/users/31b6616a1d8141d285ca9f08d207d501` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:54.950877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88142e35-bca0-4f35-9065-d6d59135a448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31b6616a1d8141d285ca9f08d207d501)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:54.953265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88142e35-bca0-4f35-9065-d6d59135a448 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:55.032859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:55.075645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-88142e35-bca0-4f35-9065-d6d59135a448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31b6616a1d8141d285ca9f08d207d501'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:55.084780 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-88142e35-bca0-4f35-9065-d6d59135a448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31b6616a1d8141d285ca9f08d207d501'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:55.085428 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-88142e35-bca0-4f35-9065-d6d59135a448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31b6616a1d8141d285ca9f08d207d501'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:55.102407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-88142e35-bca0-4f35-9065-d6d59135a448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31b6616a1d8141d285ca9f08d207d501'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:55.103761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-88142e35-bca0-4f35-9065-d6d59135a448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31b6616a1d8141d285ca9f08d207d501'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:55.132403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 985/1953] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:34:54 2022] DELETE /identity/v3/users/31b6616a1d8141d285ca9f08d207d501 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:55.516679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:55.519183 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:55.541989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8513dbb-5ae8-479f-b0bd-762b3f85270b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:55.632466 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc671528-1f54-4fd3-97a6-d24b55b9deb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:55.633884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc671528-1f54-4fd3-97a6-d24b55b9deb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:55.634378 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc671528-1f54-4fd3-97a6-d24b55b9deb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:55.634813 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc671528-1f54-4fd3-97a6-d24b55b9deb7 admin admin] PATH_INFO: `/v3/projects/a79b24f5e26e487fa667e0b328ae1709` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:55.637723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc671528-1f54-4fd3-97a6-d24b55b9deb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a79b24f5e26e487fa667e0b328ae1709)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:55.639976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc671528-1f54-4fd3-97a6-d24b55b9deb7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:55.709414 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bc671528-1f54-4fd3-97a6-d24b55b9deb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a79b24f5e26e487fa667e0b328ae1709', 'request_id': 'req-bc671528-1f54-4fd3-97a6-d24b55b9deb7'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:55.716341 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bc671528-1f54-4fd3-97a6-d24b55b9deb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a79b24f5e26e487fa667e0b328ae1709', 'request_id': 'req-bc671528-1f54-4fd3-97a6-d24b55b9deb7'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:55.726290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bc671528-1f54-4fd3-97a6-d24b55b9deb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a79b24f5e26e487fa667e0b328ae1709', 'request_id': 'req-bc671528-1f54-4fd3-97a6-d24b55b9deb7'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:55.742796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bc671528-1f54-4fd3-97a6-d24b55b9deb7 admin admin] The token cache is being invalidate because project a79b24f5e26e487fa667e0b328ae1709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:34:55.743148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bc671528-1f54-4fd3-97a6-d24b55b9deb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:55.743771 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bc671528-1f54-4fd3-97a6-d24b55b9deb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:34:55.746053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 969/1954] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:34:55 2022] DELETE /identity/v3/projects/a79b24f5e26e487fa667e0b328ae1709 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:55.752101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb06a24-b14b-4cdb-b76e-ce0e0204f4bf None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:55.752399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb06a24-b14b-4cdb-b76e-ce0e0204f4bf None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:55.752753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb06a24-b14b-4cdb-b76e-ce0e0204f4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:55.775093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7bb06a24-b14b-4cdb-b76e-ce0e0204f4bf None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:55.777814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7bb06a24-b14b-4cdb-b76e-ce0e0204f4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:55.849594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 986/1955] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:55.860252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acd361-42f1-46fa-bd70-608cc2e1ad25 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:55.860782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acd361-42f1-46fa-bd70-608cc2e1ad25 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:55.861036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acd361-42f1-46fa-bd70-608cc2e1ad25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:55.886769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e1acd361-42f1-46fa-bd70-608cc2e1ad25 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:55.888294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e1acd361-42f1-46fa-bd70-608cc2e1ad25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:55.898147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 970/1956] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:55.903192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb06a24-b14b-4cdb-b76e-ce0e0204f4bf None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:55.914699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb06a24-b14b-4cdb-b76e-ce0e0204f4bf None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:55.919949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fc5ca-69d7-4303-b332-118ef8f20be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:55.920678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fc5ca-69d7-4303-b332-118ef8f20be6 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:55.920946 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fc5ca-69d7-4303-b332-118ef8f20be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:55.921225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fc5ca-69d7-4303-b332-118ef8f20be6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:55.922022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fc5ca-69d7-4303-b332-118ef8f20be6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:55.922336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fc5ca-69d7-4303-b332-118ef8f20be6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:55.925280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fc5ca-69d7-4303-b332-118ef8f20be6 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:55.938405 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 987/1957] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:55.945681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acd361-42f1-46fa-bd70-608cc2e1ad25 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:55.955983 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acd361-42f1-46fa-bd70-608cc2e1ad25 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:55.960901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7b5d3-4581-4d0d-a218-2c1bbbb04a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:55.961576 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7b5d3-4581-4d0d-a218-2c1bbbb04a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:55.961857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7b5d3-4581-4d0d-a218-2c1bbbb04a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:55.962129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7b5d3-4581-4d0d-a218-2c1bbbb04a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:55.962842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de7b5d3-4581-4d0d-a218-2c1bbbb04a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:55.965854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de7b5d3-4581-4d0d-a218-2c1bbbb04a5b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.019259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 971/1958] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:55 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:56.026875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fc5ca-69d7-4303-b332-118ef8f20be6 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.045295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fc5ca-69d7-4303-b332-118ef8f20be6 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.173836 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0f152-95ff-4cd1-be3c-d7ecef838224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.174806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0f152-95ff-4cd1-be3c-d7ecef838224 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.175203 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0f152-95ff-4cd1-be3c-d7ecef838224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.175692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0f152-95ff-4cd1-be3c-d7ecef838224 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.176631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0f152-95ff-4cd1-be3c-d7ecef838224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.179895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0f152-95ff-4cd1-be3c-d7ecef838224 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.226158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 988/1959] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:56 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:56.231959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7b5d3-4581-4d0d-a218-2c1bbbb04a5b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.243753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7b5d3-4581-4d0d-a218-2c1bbbb04a5b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.255201 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787cac1-1832-445a-95dd-439600638b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.256684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787cac1-1832-445a-95dd-439600638b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.257244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787cac1-1832-445a-95dd-439600638b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.258325 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787cac1-1832-445a-95dd-439600638b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.259596 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787cac1-1832-445a-95dd-439600638b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.264143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787cac1-1832-445a-95dd-439600638b5b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.288964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 972/1960] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:56.307741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0f152-95ff-4cd1-be3c-d7ecef838224 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.326053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0f152-95ff-4cd1-be3c-d7ecef838224 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.337260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8c28e-2afd-46db-9a3e-2a7719abfe42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.338781 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8c28e-2afd-46db-9a3e-2a7719abfe42 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.339537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8c28e-2afd-46db-9a3e-2a7719abfe42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.340128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8c28e-2afd-46db-9a3e-2a7719abfe42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.341603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8c28e-2afd-46db-9a3e-2a7719abfe42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.344745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8c28e-2afd-46db-9a3e-2a7719abfe42 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.357604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 989/1961] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:34:56.362520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787cac1-1832-445a-95dd-439600638b5b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.373881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787cac1-1832-445a-95dd-439600638b5b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.379738 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c194c4-02d0-47be-a563-f6451f2d15fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.380708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c194c4-02d0-47be-a563-f6451f2d15fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.380996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c194c4-02d0-47be-a563-f6451f2d15fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.381278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c194c4-02d0-47be-a563-f6451f2d15fe admin admin] PATH_INFO: `/v3/projects/c539f588a41c494495fd5cbd69de9ab3/users/be3897f9c9b54a88b9a6752a83224561/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.400892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c194c4-02d0-47be-a563-f6451f2d15fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c539f588a41c494495fd5cbd69de9ab3, user_id=be3897f9c9b54a88b9a6752a83224561, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.404493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c194c4-02d0-47be-a563-f6451f2d15fe admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.421921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 973/1962] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:56 2022] PUT /identity/v3/projects/c539f588a41c494495fd5cbd69de9ab3/users/be3897f9c9b54a88b9a6752a83224561/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:56.434012 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54b3a0-712e-4c32-8903-57ba882ae8c4 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.434287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54b3a0-712e-4c32-8903-57ba882ae8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.434647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54b3a0-712e-4c32-8903-57ba882ae8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.455740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:56.468054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-fa54b3a0-712e-4c32-8903-57ba882ae8c4 None None] MFA Rules not processed for user `be3897f9c9b54a88b9a6752a83224561`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:56.469647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fa54b3a0-712e-4c32-8903-57ba882ae8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:56.543294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 990/1963] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:56.547930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c194c4-02d0-47be-a563-f6451f2d15fe admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.563856 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:56.649383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c194c4-02d0-47be-a563-f6451f2d15fe admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.655614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e1950-7a27-4c08-97b1-b4f7afdbb794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.656307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e1950-7a27-4c08-97b1-b4f7afdbb794 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.656631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e1950-7a27-4c08-97b1-b4f7afdbb794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.656910 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e1950-7a27-4c08-97b1-b4f7afdbb794 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.657628 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4e1950-7a27-4c08-97b1-b4f7afdbb794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.659552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4e1950-7a27-4c08-97b1-b4f7afdbb794 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.689195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 974/1964] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:34:56 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:56.693743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54b3a0-712e-4c32-8903-57ba882ae8c4 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.708025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54b3a0-712e-4c32-8903-57ba882ae8c4 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.769018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499458f-c707-4cce-8f30-d8463392ef33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.769697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499458f-c707-4cce-8f30-d8463392ef33 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.769978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499458f-c707-4cce-8f30-d8463392ef33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.770252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499458f-c707-4cce-8f30-d8463392ef33 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.770951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499458f-c707-4cce-8f30-d8463392ef33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.772907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499458f-c707-4cce-8f30-d8463392ef33 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.794080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 991/1965] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:34:56 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:56.798256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e1950-7a27-4c08-97b1-b4f7afdbb794 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.809456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e1950-7a27-4c08-97b1-b4f7afdbb794 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.814147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9afc2-2d13-4c45-830a-6f38a54d3cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.814952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9afc2-2d13-4c45-830a-6f38a54d3cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.815139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9afc2-2d13-4c45-830a-6f38a54d3cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.815398 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9afc2-2d13-4c45-830a-6f38a54d3cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.816189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9afc2-2d13-4c45-830a-6f38a54d3cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.818048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9afc2-2d13-4c45-830a-6f38a54d3cbd admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.831544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 975/1966] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:56.836243 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499458f-c707-4cce-8f30-d8463392ef33 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.847218 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499458f-c707-4cce-8f30-d8463392ef33 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.852050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da4d29-58b3-4f5f-a402-543fb598de78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.853013 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da4d29-58b3-4f5f-a402-543fb598de78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.853301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da4d29-58b3-4f5f-a402-543fb598de78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.853584 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da4d29-58b3-4f5f-a402-543fb598de78 admin admin] PATH_INFO: `/v3/projects/446f0583ba4c443f84a384c6b8bb35b4/users/d080961016ba4b1ca5158127ebe400ea/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.871591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da4d29-58b3-4f5f-a402-543fb598de78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=446f0583ba4c443f84a384c6b8bb35b4, user_id=d080961016ba4b1ca5158127ebe400ea, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.874559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da4d29-58b3-4f5f-a402-543fb598de78 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.889864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 992/1967] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:56 2022] PUT /identity/v3/projects/446f0583ba4c443f84a384c6b8bb35b4/users/d080961016ba4b1ca5158127ebe400ea/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:56.893846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9afc2-2d13-4c45-830a-6f38a54d3cbd admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.905331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9afc2-2d13-4c45-830a-6f38a54d3cbd admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.965458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa189c-6ea4-4277-8591-0451905b8a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.966125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa189c-6ea4-4277-8591-0451905b8a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.966415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa189c-6ea4-4277-8591-0451905b8a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.966689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa189c-6ea4-4277-8591-0451905b8a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.967426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa189c-6ea4-4277-8591-0451905b8a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.969147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa189c-6ea4-4277-8591-0451905b8a75 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.981055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 976/1968] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:34:56.985477 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da4d29-58b3-4f5f-a402-543fb598de78 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.996618 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da4d29-58b3-4f5f-a402-543fb598de78 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:57.002298 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d65015-6975-472e-9a55-21e6260fc858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:57.003283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d65015-6975-472e-9a55-21e6260fc858 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:57.003630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d65015-6975-472e-9a55-21e6260fc858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:57.004016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d65015-6975-472e-9a55-21e6260fc858 admin admin] PATH_INFO: `/v3/projects/446f0583ba4c443f84a384c6b8bb35b4/users/d080961016ba4b1ca5158127ebe400ea/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:57.006101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d65015-6975-472e-9a55-21e6260fc858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=446f0583ba4c443f84a384c6b8bb35b4, user_id=d080961016ba4b1ca5158127ebe400ea, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:57.009044 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d65015-6975-472e-9a55-21e6260fc858 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:57.021051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 993/1969] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:34:56 2022] PUT /identity/v3/projects/446f0583ba4c443f84a384c6b8bb35b4/users/d080961016ba4b1ca5158127ebe400ea/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:57.034878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d7368-6140-4694-b7dc-54703709dc46 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:57.035166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d7368-6140-4694-b7dc-54703709dc46 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:57.035447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d7368-6140-4694-b7dc-54703709dc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:57.062637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-143d7368-6140-4694-b7dc-54703709dc46 None None] MFA Rules not processed for user `d080961016ba4b1ca5158127ebe400ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:57.064015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-143d7368-6140-4694-b7dc-54703709dc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:57.137483 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 977/1970] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:34:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:34:57.148654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d65015-6975-472e-9a55-21e6260fc858 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:57.150559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f8d65015-6975-472e-9a55-21e6260fc858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:57.218459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d65015-6975-472e-9a55-21e6260fc858 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:57.222299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f561aab-ef7d-409d-9e72-6804281c93b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:57.222965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f561aab-ef7d-409d-9e72-6804281c93b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:57.223249 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f561aab-ef7d-409d-9e72-6804281c93b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:57.223534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f561aab-ef7d-409d-9e72-6804281c93b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:57.230195 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:57.235166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f561aab-ef7d-409d-9e72-6804281c93b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:57.237369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f561aab-ef7d-409d-9e72-6804281c93b8 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:57.247601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 994/1971] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:34:57.471912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d7368-6140-4694-b7dc-54703709dc46 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:57.483736 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d7368-6140-4694-b7dc-54703709dc46 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:57.487824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9cc0b-52f7-4a03-b8a0-fded1d9bd5c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:57.488510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9cc0b-52f7-4a03-b8a0-fded1d9bd5c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:57.488839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9cc0b-52f7-4a03-b8a0-fded1d9bd5c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:57.489165 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9cc0b-52f7-4a03-b8a0-fded1d9bd5c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:57.494469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:57.565219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9cc0b-52f7-4a03-b8a0-fded1d9bd5c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:57.566759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9cc0b-52f7-4a03-b8a0-fded1d9bd5c7 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:57.576530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 978/1972] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:34:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:00.843505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f561aab-ef7d-409d-9e72-6804281c93b8 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.844983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2f561aab-ef7d-409d-9e72-6804281c93b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:00.927770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f561aab-ef7d-409d-9e72-6804281c93b8 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.946538 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.948867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.949377 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.949889 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] PATH_INFO: `/v3/users/8ffcf6bceb0f4d1c93745dac18b3bf8c` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.952114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ffcf6bceb0f4d1c93745dac18b3bf8c)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:00.955762 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.038080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ffcf6bceb0f4d1c93745dac18b3bf8c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.060536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ffcf6bceb0f4d1c93745dac18b3bf8c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.062158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ffcf6bceb0f4d1c93745dac18b3bf8c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.076566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ffcf6bceb0f4d1c93745dac18b3bf8c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.077421 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ffcf6bceb0f4d1c93745dac18b3bf8c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.112522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 995/1973] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:00 2022] DELETE /identity/v3/users/8ffcf6bceb0f4d1c93745dac18b3bf8c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:01.116534 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06c6f7-3361-4501-be18-51a9afe7ad11 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.117059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06c6f7-3361-4501-be18-51a9afe7ad11 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.117206 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06c6f7-3361-4501-be18-51a9afe7ad11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.137948 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2d06c6f7-3361-4501-be18-51a9afe7ad11 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:01.139541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2d06c6f7-3361-4501-be18-51a9afe7ad11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:01.221974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 979/1974] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:01.233459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.234864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:01.304593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517fb4d-20d7-404e-9438-36eaf0e9a262 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.309021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e3baa-26bd-4582-adc1-d5e5b9779224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.309703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e3baa-26bd-4582-adc1-d5e5b9779224 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.310005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e3baa-26bd-4582-adc1-d5e5b9779224 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.310282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e3baa-26bd-4582-adc1-d5e5b9779224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.322438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e3baa-26bd-4582-adc1-d5e5b9779224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.324749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e3baa-26bd-4582-adc1-d5e5b9779224 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.339602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 996/1975] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:01.663692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06c6f7-3361-4501-be18-51a9afe7ad11 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.666241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2d06c6f7-3361-4501-be18-51a9afe7ad11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:01.687104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06c6f7-3361-4501-be18-51a9afe7ad11 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.693224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.694209 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.694560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.694907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] PATH_INFO: `/v3/projects/61cede8d05234ff09c4f13b95d8189b7` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.696152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61cede8d05234ff09c4f13b95d8189b7)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.699956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.721276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e3baa-26bd-4582-adc1-d5e5b9779224 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.723755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-446e3baa-26bd-4582-adc1-d5e5b9779224 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:01.741243 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e3baa-26bd-4582-adc1-d5e5b9779224 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.749384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6edc8-c52a-487e-b053-2c9c7449f161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.750782 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6edc8-c52a-487e-b053-2c9c7449f161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.751161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6edc8-c52a-487e-b053-2c9c7449f161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.751471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6edc8-c52a-487e-b053-2c9c7449f161 admin admin] PATH_INFO: `/v3/users/be3897f9c9b54a88b9a6752a83224561` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.752798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6edc8-c52a-487e-b053-2c9c7449f161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be3897f9c9b54a88b9a6752a83224561)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.754953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6edc8-c52a-487e-b053-2c9c7449f161 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.767108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61cede8d05234ff09c4f13b95d8189b7', 'request_id': 'req-2f614bca-558c-430f-a730-ac2771a02245'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.769320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:01.773215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61cede8d05234ff09c4f13b95d8189b7', 'request_id': 'req-2f614bca-558c-430f-a730-ac2771a02245'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.783956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61cede8d05234ff09c4f13b95d8189b7', 'request_id': 'req-2f614bca-558c-430f-a730-ac2771a02245'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.799569 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] The token cache is being invalidate because project 61cede8d05234ff09c4f13b95d8189b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:01.799890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.800790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.802588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 980/1976] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:01 2022] DELETE /identity/v3/projects/61cede8d05234ff09c4f13b95d8189b7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:01.807754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.809508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:01.826949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f6c6edc8-c52a-487e-b053-2c9c7449f161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be3897f9c9b54a88b9a6752a83224561'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.838858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f6c6edc8-c52a-487e-b053-2c9c7449f161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be3897f9c9b54a88b9a6752a83224561'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.839906 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f6c6edc8-c52a-487e-b053-2c9c7449f161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be3897f9c9b54a88b9a6752a83224561'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.846136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:01.853595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f6c6edc8-c52a-487e-b053-2c9c7449f161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be3897f9c9b54a88b9a6752a83224561'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.855430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f6c6edc8-c52a-487e-b053-2c9c7449f161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be3897f9c9b54a88b9a6752a83224561'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:01.886443 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 997/1977] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:01 2022] DELETE /identity/v3/users/be3897f9c9b54a88b9a6752a83224561 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:01.887312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f614bca-558c-430f-a730-ac2771a02245 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.894382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcdf61-672b-4a83-9733-3a50a9736d9b None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.894699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcdf61-672b-4a83-9733-3a50a9736d9b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.894986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcdf61-672b-4a83-9733-3a50a9736d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.914734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-05dcdf61-672b-4a83-9733-3a50a9736d9b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:01.915990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-05dcdf61-672b-4a83-9733-3a50a9736d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:01.960387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.961276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.961555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.961799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] PATH_INFO: `/v3/users/235f70bfd4d04298aa1604483dbc7344` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.963898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=235f70bfd4d04298aa1604483dbc7344)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.967376 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:02.011217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 998/1978] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:02.020087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcdf61-672b-4a83-9733-3a50a9736d9b None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.024527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-05dcdf61-672b-4a83-9733-3a50a9736d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:02.047705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235f70bfd4d04298aa1604483dbc7344'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:02.058469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235f70bfd4d04298aa1604483dbc7344'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:02.081410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235f70bfd4d04298aa1604483dbc7344'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:02.107241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235f70bfd4d04298aa1604483dbc7344'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:02.107954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235f70bfd4d04298aa1604483dbc7344'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:02.126718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 981/1979] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:01 2022] DELETE /identity/v3/users/235f70bfd4d04298aa1604483dbc7344 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:02.127657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcdf61-672b-4a83-9733-3a50a9736d9b None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:02.217549 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:02.231004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ce078-6ebd-4763-bd04-4474e101bceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:02.233713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ce078-6ebd-4763-bd04-4474e101bceb service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:02.236031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ce078-6ebd-4763-bd04-4474e101bceb service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:02.236318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ce078-6ebd-4763-bd04-4474e101bceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:02.238596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-972ce078-6ebd-4763-bd04-4474e101bceb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:02.382212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ce078-6ebd-4763-bd04-4474e101bceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:02.384041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ce078-6ebd-4763-bd04-4474e101bceb service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:02.405573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 999/1980] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:02.505031 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.508061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:02.524958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef4cb2-7ae7-4eb1-ac07-2bb960111e32 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:02.533702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:02.534627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:02.534914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:02.535193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] PATH_INFO: `/v3/users/253774f720204f9e812b0c5b1f3e1eb1` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:02.537883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253774f720204f9e812b0c5b1f3e1eb1)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:02.541311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:02.667103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253774f720204f9e812b0c5b1f3e1eb1'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:02.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:02.687285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253774f720204f9e812b0c5b1f3e1eb1'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:02.717612 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253774f720204f9e812b0c5b1f3e1eb1'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:02.729296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253774f720204f9e812b0c5b1f3e1eb1'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:02.730660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253774f720204f9e812b0c5b1f3e1eb1'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:02.738319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 982/1981] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:02 2022] DELETE /identity/v3/users/253774f720204f9e812b0c5b1f3e1eb1 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:02.745078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12827bef-bae1-4f0f-a6ef-b8ee79fa53c9 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:02.745672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12827bef-bae1-4f0f-a6ef-b8ee79fa53c9 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:02.746059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12827bef-bae1-4f0f-a6ef-b8ee79fa53c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:02.767996 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-12827bef-bae1-4f0f-a6ef-b8ee79fa53c9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:02.769480 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-12827bef-bae1-4f0f-a6ef-b8ee79fa53c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:02.827897 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.830099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:02.905577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1000/1982] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:02.913800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12827bef-bae1-4f0f-a6ef-b8ee79fa53c9 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.915708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-12827bef-bae1-4f0f-a6ef-b8ee79fa53c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:02.957537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9191154-c75f-4700-99e1-c8708b5f63c6 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:02.964114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dcdba-822a-4672-bdc3-796763c6b7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:02.965773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dcdba-822a-4672-bdc3-796763c6b7a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:02.966141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dcdba-822a-4672-bdc3-796763c6b7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:02.966574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dcdba-822a-4672-bdc3-796763c6b7a1 admin admin] PATH_INFO: `/v3/projects/76d27dab0d14438a8714eb2e94257630` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:02.968935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833dcdba-822a-4672-bdc3-796763c6b7a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d27dab0d14438a8714eb2e94257630)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:02.971788 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833dcdba-822a-4672-bdc3-796763c6b7a1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.009049 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12827bef-bae1-4f0f-a6ef-b8ee79fa53c9 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.016525 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22235087-9b36-4ecd-9645-1f11fe08482d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.018127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22235087-9b36-4ecd-9645-1f11fe08482d service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.018572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22235087-9b36-4ecd-9645-1f11fe08482d service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.018856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22235087-9b36-4ecd-9645-1f11fe08482d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.033636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22235087-9b36-4ecd-9645-1f11fe08482d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.035325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22235087-9b36-4ecd-9645-1f11fe08482d service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.039928 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-833dcdba-822a-4672-bdc3-796763c6b7a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d27dab0d14438a8714eb2e94257630', 'request_id': 'req-833dcdba-822a-4672-bdc3-796763c6b7a1'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.047415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-833dcdba-822a-4672-bdc3-796763c6b7a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d27dab0d14438a8714eb2e94257630', 'request_id': 'req-833dcdba-822a-4672-bdc3-796763c6b7a1'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.048355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1001/1983] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:03.049282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22235087-9b36-4ecd-9645-1f11fe08482d service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.050695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-22235087-9b36-4ecd-9645-1f11fe08482d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:03.057140 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-833dcdba-822a-4672-bdc3-796763c6b7a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d27dab0d14438a8714eb2e94257630', 'request_id': 'req-833dcdba-822a-4672-bdc3-796763c6b7a1'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.066782 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22235087-9b36-4ecd-9645-1f11fe08482d service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.073420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-833dcdba-822a-4672-bdc3-796763c6b7a1 admin admin] The token cache is being invalidate because project 76d27dab0d14438a8714eb2e94257630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:03.073800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-833dcdba-822a-4672-bdc3-796763c6b7a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.074576 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-833dcdba-822a-4672-bdc3-796763c6b7a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.076406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 983/1984] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:02 2022] DELETE /identity/v3/projects/76d27dab0d14438a8714eb2e94257630 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:03.081221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808c478-a98f-4a9f-a576-e7b709d9fd4b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.081550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808c478-a98f-4a9f-a576-e7b709d9fd4b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.081837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808c478-a98f-4a9f-a576-e7b709d9fd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.109060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8808c478-a98f-4a9f-a576-e7b709d9fd4b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:03.111857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8808c478-a98f-4a9f-a576-e7b709d9fd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:03.125419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:03.182757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.184331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.185548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.186106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] PATH_INFO: `/v3/projects/c539f588a41c494495fd5cbd69de9ab3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.188101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c539f588a41c494495fd5cbd69de9ab3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.190640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.192347 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:03.208844 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 984/1985] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:03.218559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a06bed-8390-4a2b-8936-987d21df48da None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.219173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a06bed-8390-4a2b-8936-987d21df48da None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.219869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a06bed-8390-4a2b-8936-987d21df48da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.257522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-87a06bed-8390-4a2b-8936-987d21df48da None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:03.259354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-87a06bed-8390-4a2b-8936-987d21df48da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:03.280562 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c539f588a41c494495fd5cbd69de9ab3', 'request_id': 'req-a38d49bc-ea09-4bb9-b371-496ebb478b67'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.295298 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c539f588a41c494495fd5cbd69de9ab3', 'request_id': 'req-a38d49bc-ea09-4bb9-b371-496ebb478b67'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.311904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c539f588a41c494495fd5cbd69de9ab3', 'request_id': 'req-a38d49bc-ea09-4bb9-b371-496ebb478b67'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.330665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] The token cache is being invalidate because project c539f588a41c494495fd5cbd69de9ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:03.331108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.334954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.336751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1002/1986] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:03 2022] DELETE /identity/v3/projects/c539f588a41c494495fd5cbd69de9ab3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:03.337783 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.340020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:03.349946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 985/1987] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:03.354645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a06bed-8390-4a2b-8936-987d21df48da None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.364910 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a06bed-8390-4a2b-8936-987d21df48da None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.375732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3da095-a055-4223-9b30-f12f6f4053fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.376994 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3da095-a055-4223-9b30-f12f6f4053fc admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.377290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3da095-a055-4223-9b30-f12f6f4053fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.377576 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3da095-a055-4223-9b30-f12f6f4053fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.378365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3da095-a055-4223-9b30-f12f6f4053fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:03.378680 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3da095-a055-4223-9b30-f12f6f4053fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.380986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3da095-a055-4223-9b30-f12f6f4053fc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.400653 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 986/1988] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:03.403804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3da095-a055-4223-9b30-f12f6f4053fc admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.418126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3da095-a055-4223-9b30-f12f6f4053fc admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.421387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12eebd-6635-4e54-8f77-efa4b06c0753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.422090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12eebd-6635-4e54-8f77-efa4b06c0753 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.422391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12eebd-6635-4e54-8f77-efa4b06c0753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.422675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12eebd-6635-4e54-8f77-efa4b06c0753 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.423794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12eebd-6635-4e54-8f77-efa4b06c0753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.425859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12eebd-6635-4e54-8f77-efa4b06c0753 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.460873 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 987/1989] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:03 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:03.466443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12eebd-6635-4e54-8f77-efa4b06c0753 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.481413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d49bc-ea09-4bb9-b371-496ebb478b67 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.568425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.569513 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.569799 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.570079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] PATH_INFO: `/v3/users/d080961016ba4b1ca5158127ebe400ea` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.571430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d080961016ba4b1ca5158127ebe400ea)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.574694 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12eebd-6635-4e54-8f77-efa4b06c0753 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.574827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.583752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2612bc-f311-4eac-a3cc-0ba33174661a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.584595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2612bc-f311-4eac-a3cc-0ba33174661a admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.584954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2612bc-f311-4eac-a3cc-0ba33174661a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.585346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2612bc-f311-4eac-a3cc-0ba33174661a admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.586125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2612bc-f311-4eac-a3cc-0ba33174661a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.589352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2612bc-f311-4eac-a3cc-0ba33174661a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.605757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:03.620334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 988/1990] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:03 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:03.624456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2612bc-f311-4eac-a3cc-0ba33174661a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.627004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0b2612bc-f311-4eac-a3cc-0ba33174661a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:03.643221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2612bc-f311-4eac-a3cc-0ba33174661a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.648514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.649396 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.649681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.649956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] PATH_INFO: `/v3/projects/8ba05be1e1da46d395460dc420528e2e` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.652416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba05be1e1da46d395460dc420528e2e)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.654816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.679669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd080961016ba4b1ca5158127ebe400ea'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.702386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd080961016ba4b1ca5158127ebe400ea'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.703624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd080961016ba4b1ca5158127ebe400ea'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.716284 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd080961016ba4b1ca5158127ebe400ea'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.717346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd080961016ba4b1ca5158127ebe400ea'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.742227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:03.749577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba05be1e1da46d395460dc420528e2e', 'request_id': 'req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.758755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1003/1991] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:03 2022] DELETE /identity/v3/users/d080961016ba4b1ca5158127ebe400ea => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:03.759973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.762237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:03.773399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba05be1e1da46d395460dc420528e2e', 'request_id': 'req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.786801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba05be1e1da46d395460dc420528e2e', 'request_id': 'req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.811229 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] The token cache is being invalidate because project 8ba05be1e1da46d395460dc420528e2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:03.811894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.813083 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:03.819821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 989/1992] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:03 2022] DELETE /identity/v3/projects/8ba05be1e1da46d395460dc420528e2e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:03.822010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.823767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:04.008526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2f071-eb0a-455f-b7c0-0cc40df83c41 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.060692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de55d5-32ab-4392-9c21-642b03b92ccd admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.075798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779bd59-c333-4eeb-b7c6-52304e03d81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.077244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779bd59-c333-4eeb-b7c6-52304e03d81d admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.077741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779bd59-c333-4eeb-b7c6-52304e03d81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.078260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779bd59-c333-4eeb-b7c6-52304e03d81d admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.079818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779bd59-c333-4eeb-b7c6-52304e03d81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.082065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779bd59-c333-4eeb-b7c6-52304e03d81d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.113618 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1004/1993] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:04.126362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779bd59-c333-4eeb-b7c6-52304e03d81d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:04.131168 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9779bd59-c333-4eeb-b7c6-52304e03d81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:04.155489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.156656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.157064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.157507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] PATH_INFO: `/v3/users/9cf001c0c1584bfb83287e5ef5500324` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.162227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cf001c0c1584bfb83287e5ef5500324)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.164624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.164825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779bd59-c333-4eeb-b7c6-52304e03d81d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.188113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322ffea-5098-46a2-91bf-601a5d4f9512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.189024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322ffea-5098-46a2-91bf-601a5d4f9512 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.189188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322ffea-5098-46a2-91bf-601a5d4f9512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.189401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322ffea-5098-46a2-91bf-601a5d4f9512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.190201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322ffea-5098-46a2-91bf-601a5d4f9512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.191871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322ffea-5098-46a2-91bf-601a5d4f9512 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.205999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1005/1994] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:04.213449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322ffea-5098-46a2-91bf-601a5d4f9512 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:04.227401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322ffea-5098-46a2-91bf-601a5d4f9512 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.241158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4714749-1435-4b1f-be7b-3b1dfc7cb52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.242102 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4714749-1435-4b1f-be7b-3b1dfc7cb52a admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.242395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4714749-1435-4b1f-be7b-3b1dfc7cb52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.242682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4714749-1435-4b1f-be7b-3b1dfc7cb52a admin admin] PATH_INFO: `/v3/projects/3e697ff997e04921ab46574cc830ab0f/users/8e4225954fc443e0a0413fd3c4d3871e/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.253912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cf001c0c1584bfb83287e5ef5500324'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:04.265210 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cf001c0c1584bfb83287e5ef5500324'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:04.269851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4714749-1435-4b1f-be7b-3b1dfc7cb52a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e697ff997e04921ab46574cc830ab0f, user_id=8e4225954fc443e0a0413fd3c4d3871e, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.273896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4714749-1435-4b1f-be7b-3b1dfc7cb52a admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.298490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cf001c0c1584bfb83287e5ef5500324'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:04.312160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1006/1995] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:04 2022] PUT /identity/v3/projects/3e697ff997e04921ab46574cc830ab0f/users/8e4225954fc443e0a0413fd3c4d3871e/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:04.313930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96801c12-663b-41dc-8d03-f98074e6cc7f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.314238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96801c12-663b-41dc-8d03-f98074e6cc7f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.314494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96801c12-663b-41dc-8d03-f98074e6cc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.316702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cf001c0c1584bfb83287e5ef5500324'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:04.317898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cf001c0c1584bfb83287e5ef5500324'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:04.327373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 990/1996] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:03 2022] DELETE /identity/v3/users/9cf001c0c1584bfb83287e5ef5500324 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:04.356124 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-96801c12-663b-41dc-8d03-f98074e6cc7f None None] MFA Rules not processed for user `8e4225954fc443e0a0413fd3c4d3871e`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:04.357487 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-96801c12-663b-41dc-8d03-f98074e6cc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:04.359136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:04.360643 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:04.366861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:04.438317 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:04.481114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1007/1997] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:04.488200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96801c12-663b-41dc-8d03-f98074e6cc7f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:04.490304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-96801c12-663b-41dc-8d03-f98074e6cc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:04.493924 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a0942-5e2e-4d07-a449-d1a0c2ee37fe admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.505263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925864f-86f7-471d-b8c8-8d1a07388cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.506395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925864f-86f7-471d-b8c8-8d1a07388cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.508189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925864f-86f7-471d-b8c8-8d1a07388cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.508514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925864f-86f7-471d-b8c8-8d1a07388cd6 admin admin] PATH_INFO: `/v3/projects/446f0583ba4c443f84a384c6b8bb35b4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.513281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925864f-86f7-471d-b8c8-8d1a07388cd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=446f0583ba4c443f84a384c6b8bb35b4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.515277 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925864f-86f7-471d-b8c8-8d1a07388cd6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.519312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96801c12-663b-41dc-8d03-f98074e6cc7f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.526375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2e713-a082-4615-9f18-1352c6e95977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.526664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2e713-a082-4615-9f18-1352c6e95977 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.526851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2e713-a082-4615-9f18-1352c6e95977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.527417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2e713-a082-4615-9f18-1352c6e95977 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.527922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2e713-a082-4615-9f18-1352c6e95977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.530006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2e713-a082-4615-9f18-1352c6e95977 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.602437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1008/1998] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:04 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:04.610958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2e713-a082-4615-9f18-1352c6e95977 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:04.661204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e925864f-86f7-471d-b8c8-8d1a07388cd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '446f0583ba4c443f84a384c6b8bb35b4', 'request_id': 'req-e925864f-86f7-471d-b8c8-8d1a07388cd6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:04.672839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e925864f-86f7-471d-b8c8-8d1a07388cd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '446f0583ba4c443f84a384c6b8bb35b4', 'request_id': 'req-e925864f-86f7-471d-b8c8-8d1a07388cd6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:04.690961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e925864f-86f7-471d-b8c8-8d1a07388cd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '446f0583ba4c443f84a384c6b8bb35b4', 'request_id': 'req-e925864f-86f7-471d-b8c8-8d1a07388cd6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:04.719491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e925864f-86f7-471d-b8c8-8d1a07388cd6 admin admin] The token cache is being invalidate because project 446f0583ba4c443f84a384c6b8bb35b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:04.719970 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e925864f-86f7-471d-b8c8-8d1a07388cd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:04.721070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e925864f-86f7-471d-b8c8-8d1a07388cd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:04.723805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 991/1999] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:04 2022] DELETE /identity/v3/projects/446f0583ba4c443f84a384c6b8bb35b4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:04.732272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08cb01-3309-48fa-b016-945bfdfa3471 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.732763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08cb01-3309-48fa-b016-945bfdfa3471 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.733203 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08cb01-3309-48fa-b016-945bfdfa3471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.739899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2e713-a082-4615-9f18-1352c6e95977 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.772896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5a08cb01-3309-48fa-b016-945bfdfa3471 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:04.774635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5a08cb01-3309-48fa-b016-945bfdfa3471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:04.851371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847c326-410f-4df5-96f4-023a8153cdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.852342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847c326-410f-4df5-96f4-023a8153cdff admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.852539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847c326-410f-4df5-96f4-023a8153cdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.853121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847c326-410f-4df5-96f4-023a8153cdff admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.858537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847c326-410f-4df5-96f4-023a8153cdff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.858537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847c326-410f-4df5-96f4-023a8153cdff admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.894052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1009/2000] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:04 2022] POST /identity/v3/users => generated 298 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:04.894052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847c326-410f-4df5-96f4-023a8153cdff admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:04.894412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-e847c326-410f-4df5-96f4-023a8153cdff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:04.902142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 992/2001] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:04.903483 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08cb01-3309-48fa-b016-945bfdfa3471 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:04.906073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5a08cb01-3309-48fa-b016-945bfdfa3471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:04.923033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08cb01-3309-48fa-b016-945bfdfa3471 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.923314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847c326-410f-4df5-96f4-023a8153cdff admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.933665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.935243 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.935780 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.936283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] PATH_INFO: `/v3/projects/17725f01b84242dba92a8727d6c9df53` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.937518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b26830-4020-4bf4-91a2-395edd5deadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.938267 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b26830-4020-4bf4-91a2-395edd5deadd admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.938552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b26830-4020-4bf4-91a2-395edd5deadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.938859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b26830-4020-4bf4-91a2-395edd5deadd admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.939805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b26830-4020-4bf4-91a2-395edd5deadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.940401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17725f01b84242dba92a8727d6c9df53)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.941677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b26830-4020-4bf4-91a2-395edd5deadd admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.944286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.953179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 993/2002] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:04.955101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a027-64e8-43fa-a7ce-36ab4b2aa0b9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.955225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a027-64e8-43fa-a7ce-36ab4b2aa0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.955561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a027-64e8-43fa-a7ce-36ab4b2aa0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.975543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-17e0a027-64e8-43fa-a7ce-36ab4b2aa0b9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:04.976811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-17e0a027-64e8-43fa-a7ce-36ab4b2aa0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:04.985857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 994/2003] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:04.986664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a027-64e8-43fa-a7ce-36ab4b2aa0b9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:04.997920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a027-64e8-43fa-a7ce-36ab4b2aa0b9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.004030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee84517-24b9-4307-b756-a5314cf1f920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.005264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee84517-24b9-4307-b756-a5314cf1f920 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.005613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee84517-24b9-4307-b756-a5314cf1f920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.005996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee84517-24b9-4307-b756-a5314cf1f920 admin admin] PATH_INFO: `/v3/projects/7cbd9edebbd049719d4738b56a29a136/users/838a69445d2041d8a994cea8fada2d66/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.024865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:05.032638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17725f01b84242dba92a8727d6c9df53', 'request_id': 'req-5bb5d818-33ae-43ca-ad0c-64e450040ca8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:05.035391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee84517-24b9-4307-b756-a5314cf1f920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbd9edebbd049719d4738b56a29a136, user_id=838a69445d2041d8a994cea8fada2d66, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.038222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:05.038959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee84517-24b9-4307-b756-a5314cf1f920 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.048031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17725f01b84242dba92a8727d6c9df53', 'request_id': 'req-5bb5d818-33ae-43ca-ad0c-64e450040ca8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:05.062338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17725f01b84242dba92a8727d6c9df53', 'request_id': 'req-5bb5d818-33ae-43ca-ad0c-64e450040ca8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:05.070581 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 995/2004] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:04 2022] PUT /identity/v3/projects/7cbd9edebbd049719d4738b56a29a136/users/838a69445d2041d8a994cea8fada2d66/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:05.071180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee84517-24b9-4307-b756-a5314cf1f920 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.078260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] The token cache is being invalidate because project 17725f01b84242dba92a8727d6c9df53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:05.078589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:05.079225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:05.080862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1010/2005] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:04 2022] DELETE /identity/v3/projects/17725f01b84242dba92a8727d6c9df53 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:05.082212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.083646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:05.167740 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee84517-24b9-4307-b756-a5314cf1f920 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.175632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b2ec4-8548-49fb-ab8f-ad9b6f5b72ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.176015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b2ec4-8548-49fb-ab8f-ad9b6f5b72ee admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.176413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b2ec4-8548-49fb-ab8f-ad9b6f5b72ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.176825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b2ec4-8548-49fb-ab8f-ad9b6f5b72ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.177740 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b2ec4-8548-49fb-ab8f-ad9b6f5b72ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:05.178868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b2ec4-8548-49fb-ab8f-ad9b6f5b72ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.179680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5d818-33ae-43ca-ad0c-64e450040ca8 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.183708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b2ec4-8548-49fb-ab8f-ad9b6f5b72ee admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.188604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a4123-6018-489a-992d-4a17afdb47f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.189312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a4123-6018-489a-992d-4a17afdb47f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.189627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a4123-6018-489a-992d-4a17afdb47f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.190372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a4123-6018-489a-992d-4a17afdb47f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.190638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a4123-6018-489a-992d-4a17afdb47f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.192936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a4123-6018-489a-992d-4a17afdb47f5 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.214755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 996/2006] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:05.214920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1011/2007] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:05.215016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea43ba-4b3c-4cbf-ab95-a3abb9731a21 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.215218 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea43ba-4b3c-4cbf-ab95-a3abb9731a21 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.216100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a4123-6018-489a-992d-4a17afdb47f5 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.216230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea43ba-4b3c-4cbf-ab95-a3abb9731a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.229607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a4123-6018-489a-992d-4a17afdb47f5 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.235966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e260d-6055-4b9c-bfc7-503853455922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.237122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e260d-6055-4b9c-bfc7-503853455922 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.237417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e260d-6055-4b9c-bfc7-503853455922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.237753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e260d-6055-4b9c-bfc7-503853455922 admin admin] PATH_INFO: `/v3/projects/7cbd9edebbd049719d4738b56a29a136/users/838a69445d2041d8a994cea8fada2d66/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.240086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e260d-6055-4b9c-bfc7-503853455922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbd9edebbd049719d4738b56a29a136, user_id=838a69445d2041d8a994cea8fada2d66, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.252263 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e260d-6055-4b9c-bfc7-503853455922 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.258841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-beea43ba-4b3c-4cbf-ab95-a3abb9731a21 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:05.262646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-beea43ba-4b3c-4cbf-ab95-a3abb9731a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:05.267665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1012/2008] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:05 2022] PUT /identity/v3/projects/7cbd9edebbd049719d4738b56a29a136/users/838a69445d2041d8a994cea8fada2d66/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:05.268247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e260d-6055-4b9c-bfc7-503853455922 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.270246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a77e260d-6055-4b9c-bfc7-503853455922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:05.396581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e260d-6055-4b9c-bfc7-503853455922 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.404853 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95ee40-f8de-4ebb-b4ee-dab5d989f7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.406966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95ee40-f8de-4ebb-b4ee-dab5d989f7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.406966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95ee40-f8de-4ebb-b4ee-dab5d989f7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.406966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95ee40-f8de-4ebb-b4ee-dab5d989f7f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.407317 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95ee40-f8de-4ebb-b4ee-dab5d989f7f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.409355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95ee40-f8de-4ebb-b4ee-dab5d989f7f5 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.414002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 997/2009] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:05.416610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da986f6c-4367-4fd1-878b-67840cc49aee None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.417291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da986f6c-4367-4fd1-878b-67840cc49aee None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.417815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da986f6c-4367-4fd1-878b-67840cc49aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.460302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1013/2010] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:05 2022] POST /identity/v3/projects => generated 361 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:05.465973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b947b-f7e9-4877-9ae8-83dc4730fff2 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.466283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b947b-f7e9-4877-9ae8-83dc4730fff2 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.466621 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b947b-f7e9-4877-9ae8-83dc4730fff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.487419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f95b947b-f7e9-4877-9ae8-83dc4730fff2 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:05.488129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-da986f6c-4367-4fd1-878b-67840cc49aee None None] MFA Rules not processed for user `838a69445d2041d8a994cea8fada2d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:05.488739 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f95b947b-f7e9-4877-9ae8-83dc4730fff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:05.490749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-da986f6c-4367-4fd1-878b-67840cc49aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:05.591399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:05.615075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1014/2011] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:05.621119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b947b-f7e9-4877-9ae8-83dc4730fff2 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.639894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b947b-f7e9-4877-9ae8-83dc4730fff2 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.657929 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47711e8-8084-41de-a6f8-1d8887c78fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.661389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47711e8-8084-41de-a6f8-1d8887c78fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.661389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47711e8-8084-41de-a6f8-1d8887c78fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.661389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47711e8-8084-41de-a6f8-1d8887c78fbe admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.661389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47711e8-8084-41de-a6f8-1d8887c78fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.662712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47711e8-8084-41de-a6f8-1d8887c78fbe admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.682588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 998/2012] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:05.682588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da986f6c-4367-4fd1-878b-67840cc49aee None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.697881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da986f6c-4367-4fd1-878b-67840cc49aee None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.706919 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3e3bd-eff2-41f2-a670-5c7d41c249e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.708031 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3e3bd-eff2-41f2-a670-5c7d41c249e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.708729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3e3bd-eff2-41f2-a670-5c7d41c249e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.709056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3e3bd-eff2-41f2-a670-5c7d41c249e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.711649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3e3bd-eff2-41f2-a670-5c7d41c249e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:05.712156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3e3bd-eff2-41f2-a670-5c7d41c249e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.714723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3e3bd-eff2-41f2-a670-5c7d41c249e1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.719964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1015/2013] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:05 2022] POST /identity/v3/users => generated 281 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:05.724580 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47711e8-8084-41de-a6f8-1d8887c78fbe admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.725046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:05.735931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 999/2014] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:05.738376 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3e3bd-eff2-41f2-a670-5c7d41c249e1 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.772669 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3e3bd-eff2-41f2-a670-5c7d41c249e1 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.779372 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebd720-39cf-433c-8e77-ff4871154db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.780057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebd720-39cf-433c-8e77-ff4871154db6 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.780340 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebd720-39cf-433c-8e77-ff4871154db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.780684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebd720-39cf-433c-8e77-ff4871154db6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.781364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebd720-39cf-433c-8e77-ff4871154db6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.782280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47711e8-8084-41de-a6f8-1d8887c78fbe admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.784009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebd720-39cf-433c-8e77-ff4871154db6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.793452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bf416-511d-4bc5-8db4-838b9a8ae355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.794981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bf416-511d-4bc5-8db4-838b9a8ae355 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.795624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bf416-511d-4bc5-8db4-838b9a8ae355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.796249 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bf416-511d-4bc5-8db4-838b9a8ae355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.797875 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0bf416-511d-4bc5-8db4-838b9a8ae355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.802821 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0bf416-511d-4bc5-8db4-838b9a8ae355 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.840412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1016/2015] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:05.847484 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1000/2016] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:05 2022] POST /identity/v3/projects => generated 361 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:05.848441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bf416-511d-4bc5-8db4-838b9a8ae355 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.849170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebd720-39cf-433c-8e77-ff4871154db6 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.851937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ce0bf416-511d-4bc5-8db4-838b9a8ae355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:05.976344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bf416-511d-4bc5-8db4-838b9a8ae355 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.061180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebd720-39cf-433c-8e77-ff4871154db6 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.066971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75697d44-11ce-42fa-bd33-e4365611f0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.067786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75697d44-11ce-42fa-bd33-e4365611f0de admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.068165 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75697d44-11ce-42fa-bd33-e4365611f0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.068685 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75697d44-11ce-42fa-bd33-e4365611f0de admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.069526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75697d44-11ce-42fa-bd33-e4365611f0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.071268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75697d44-11ce-42fa-bd33-e4365611f0de admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.083485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1001/2017] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:06.084452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75697d44-11ce-42fa-bd33-e4365611f0de admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.091679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122969b-5ea6-49c9-a9ed-b9de698023f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.093165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122969b-5ea6-49c9-a9ed-b9de698023f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.093555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122969b-5ea6-49c9-a9ed-b9de698023f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.094014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122969b-5ea6-49c9-a9ed-b9de698023f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.097181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75697d44-11ce-42fa-bd33-e4365611f0de admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.103095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8f695-34a1-46f9-bf4c-d0e6f12a477a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.104419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8f695-34a1-46f9-bf4c-d0e6f12a477a admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.104657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8f695-34a1-46f9-bf4c-d0e6f12a477a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.104978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8f695-34a1-46f9-bf4c-d0e6f12a477a admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.105761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8f695-34a1-46f9-bf4c-d0e6f12a477a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.107876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8f695-34a1-46f9-bf4c-d0e6f12a477a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.134554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1002/2018] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:06 2022] POST /identity/v3/users => generated 281 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:06.134554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8f695-34a1-46f9-bf4c-d0e6f12a477a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.143168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8f695-34a1-46f9-bf4c-d0e6f12a477a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.147966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3688c3-d3b3-4d72-9460-a1ec93d95990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.148819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3688c3-d3b3-4d72-9460-a1ec93d95990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.149106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3688c3-d3b3-4d72-9460-a1ec93d95990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.149533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3688c3-d3b3-4d72-9460-a1ec93d95990 admin admin] PATH_INFO: `/v3/projects/c2e526d5f3854195ac646a6532e4e43f/users/37bbfca892874965bbfe9ba528d8eb12/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.170790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3688c3-d3b3-4d72-9460-a1ec93d95990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e526d5f3854195ac646a6532e4e43f, user_id=37bbfca892874965bbfe9ba528d8eb12, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.173731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3688c3-d3b3-4d72-9460-a1ec93d95990 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.193735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1003/2019] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:06 2022] PUT /identity/v3/projects/c2e526d5f3854195ac646a6532e4e43f/users/37bbfca892874965bbfe9ba528d8eb12/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:06.199611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3688c3-d3b3-4d72-9460-a1ec93d95990 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.207964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122969b-5ea6-49c9-a9ed-b9de698023f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.210399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122969b-5ea6-49c9-a9ed-b9de698023f5 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.226742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1017/2020] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:06.229606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232465c-71dd-4a4a-9d70-99dbb18f9cd7 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.231339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232465c-71dd-4a4a-9d70-99dbb18f9cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.231429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232465c-71dd-4a4a-9d70-99dbb18f9cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.298397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3688c3-d3b3-4d72-9460-a1ec93d95990 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.299786 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-1232465c-71dd-4a4a-9d70-99dbb18f9cd7 None None] MFA Rules not processed for user `37bbfca892874965bbfe9ba528d8eb12`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:06.303065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1232465c-71dd-4a4a-9d70-99dbb18f9cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:06.304016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5f8d4-283f-461c-9b65-9e19c84b0b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.304764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5f8d4-283f-461c-9b65-9e19c84b0b74 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.305054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5f8d4-283f-461c-9b65-9e19c84b0b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.305473 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5f8d4-283f-461c-9b65-9e19c84b0b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.306352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5f8d4-283f-461c-9b65-9e19c84b0b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.308943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5f8d4-283f-461c-9b65-9e19c84b0b74 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.329059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1004/2021] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:06.334933 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5f8d4-283f-461c-9b65-9e19c84b0b74 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.351081 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5f8d4-283f-461c-9b65-9e19c84b0b74 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.353418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:06.359066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5469410-00d0-4c5b-a269-a090672b81c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.359950 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5469410-00d0-4c5b-a269-a090672b81c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.360244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5469410-00d0-4c5b-a269-a090672b81c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.360559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5469410-00d0-4c5b-a269-a090672b81c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.361307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5469410-00d0-4c5b-a269-a090672b81c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.363126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5469410-00d0-4c5b-a269-a090672b81c8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.379623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1005/2022] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:06.379623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5469410-00d0-4c5b-a269-a090672b81c8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.403021 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:06.446956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1018/2023] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:06.448902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232465c-71dd-4a4a-9d70-99dbb18f9cd7 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.462499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232465c-71dd-4a4a-9d70-99dbb18f9cd7 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.470847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014e5c9-05a1-4fe2-81e8-a7fe97684257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.471991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014e5c9-05a1-4fe2-81e8-a7fe97684257 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.472360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014e5c9-05a1-4fe2-81e8-a7fe97684257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.472934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014e5c9-05a1-4fe2-81e8-a7fe97684257 admin admin] PATH_INFO: `/v3/projects/56e7ba4044754f438856fb9c7c5e5b3a/users/758ab03d602043dc9d8c87bf61671db8/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.476861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5469410-00d0-4c5b-a269-a090672b81c8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.484520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe35c8-328b-4dc1-af4e-4c85e236e967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.485266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe35c8-328b-4dc1-af4e-4c85e236e967 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.485613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe35c8-328b-4dc1-af4e-4c85e236e967 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.485942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe35c8-328b-4dc1-af4e-4c85e236e967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.489429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7afe35c8-328b-4dc1-af4e-4c85e236e967 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:06.497575 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014e5c9-05a1-4fe2-81e8-a7fe97684257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e7ba4044754f438856fb9c7c5e5b3a, user_id=758ab03d602043dc9d8c87bf61671db8, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.500517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014e5c9-05a1-4fe2-81e8-a7fe97684257 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.521120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1019/2024] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:06 2022] PUT /identity/v3/projects/56e7ba4044754f438856fb9c7c5e5b3a/users/758ab03d602043dc9d8c87bf61671db8/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:06.544281 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d0aa5-35c2-40bd-96c7-cce091432c7f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.544954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d0aa5-35c2-40bd-96c7-cce091432c7f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.545604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d0aa5-35c2-40bd-96c7-cce091432c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.578264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-714d0aa5-35c2-40bd-96c7-cce091432c7f None None] MFA Rules not processed for user `758ab03d602043dc9d8c87bf61671db8`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:06.579505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-714d0aa5-35c2-40bd-96c7-cce091432c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:06.609340 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe35c8-328b-4dc1-af4e-4c85e236e967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.612386 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe35c8-328b-4dc1-af4e-4c85e236e967 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.625234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1006/2025] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:06.625959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe35c8-328b-4dc1-af4e-4c85e236e967 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.627697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7afe35c8-328b-4dc1-af4e-4c85e236e967 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:06.644066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe35c8-328b-4dc1-af4e-4c85e236e967 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.677810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1020/2026] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:06.682746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d0aa5-35c2-40bd-96c7-cce091432c7f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.734407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45331237-cd27-4d2f-89bb-9995cfbd75b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.735301 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45331237-cd27-4d2f-89bb-9995cfbd75b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.735776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45331237-cd27-4d2f-89bb-9995cfbd75b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.736152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45331237-cd27-4d2f-89bb-9995cfbd75b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.779101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d0aa5-35c2-40bd-96c7-cce091432c7f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.785624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a88773-ae57-4863-9390-dff4e45f4efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.786916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a88773-ae57-4863-9390-dff4e45f4efa admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.786916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a88773-ae57-4863-9390-dff4e45f4efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.786916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a88773-ae57-4863-9390-dff4e45f4efa admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.787593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a88773-ae57-4863-9390-dff4e45f4efa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.789611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a88773-ae57-4863-9390-dff4e45f4efa admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.835716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45331237-cd27-4d2f-89bb-9995cfbd75b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.837564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45331237-cd27-4d2f-89bb-9995cfbd75b6 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.841082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1021/2027] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:06 2022] POST /identity/v3/projects => generated 365 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:06.851483 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1007/2028] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:06.852518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a88773-ae57-4863-9390-dff4e45f4efa admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.871593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:06.876577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45331237-cd27-4d2f-89bb-9995cfbd75b6 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.887417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45331237-cd27-4d2f-89bb-9995cfbd75b6 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.935045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:06.953533 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a88773-ae57-4863-9390-dff4e45f4efa admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.959329 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48324770-6343-4802-8c31-25ffa9baf12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.960167 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48324770-6343-4802-8c31-25ffa9baf12b admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.960454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48324770-6343-4802-8c31-25ffa9baf12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.960770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48324770-6343-4802-8c31-25ffa9baf12b admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.962268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48324770-6343-4802-8c31-25ffa9baf12b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.963894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48324770-6343-4802-8c31-25ffa9baf12b admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.991059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1022/2029] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:06 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:06.997642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48324770-6343-4802-8c31-25ffa9baf12b admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.005312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.006535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.006859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.007096 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] PATH_INFO: `/v3/users/37bbfca892874965bbfe9ba528d8eb12` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.009075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37bbfca892874965bbfe9ba528d8eb12)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.011270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48324770-6343-4802-8c31-25ffa9baf12b admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.011759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.016711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8eb07c-5088-44cd-a367-02a4eb7bc3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.017617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8eb07c-5088-44cd-a367-02a4eb7bc3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.017848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8eb07c-5088-44cd-a367-02a4eb7bc3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.018118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8eb07c-5088-44cd-a367-02a4eb7bc3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.018959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8eb07c-5088-44cd-a367-02a4eb7bc3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.020702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8eb07c-5088-44cd-a367-02a4eb7bc3bd admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.045647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1023/2030] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:07.047066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8eb07c-5088-44cd-a367-02a4eb7bc3bd admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.060831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8eb07c-5088-44cd-a367-02a4eb7bc3bd admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.066057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23406d0-512f-4b58-b143-f73ee05fbda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.067220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23406d0-512f-4b58-b143-f73ee05fbda1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.067410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23406d0-512f-4b58-b143-f73ee05fbda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.067570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23406d0-512f-4b58-b143-f73ee05fbda1 admin admin] PATH_INFO: `/v3/projects/6d25fc6cb4754e7b9b2da37091351ffb/users/5920648a631d4539a6cadfb255fd7a68/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.088827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23406d0-512f-4b58-b143-f73ee05fbda1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d25fc6cb4754e7b9b2da37091351ffb, user_id=5920648a631d4539a6cadfb255fd7a68, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.091847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23406d0-512f-4b58-b143-f73ee05fbda1 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.103313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37bbfca892874965bbfe9ba528d8eb12'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:07.112622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37bbfca892874965bbfe9ba528d8eb12'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:07.120581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1024/2031] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:07 2022] PUT /identity/v3/projects/6d25fc6cb4754e7b9b2da37091351ffb/users/5920648a631d4539a6cadfb255fd7a68/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:07.121815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23406d0-512f-4b58-b143-f73ee05fbda1 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.144231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37bbfca892874965bbfe9ba528d8eb12'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:07.162138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37bbfca892874965bbfe9ba528d8eb12'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:07.163037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37bbfca892874965bbfe9ba528d8eb12'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:07.176551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1008/2032] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:06 2022] DELETE /identity/v3/users/37bbfca892874965bbfe9ba528d8eb12 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:07.177833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.179450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:07.226590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23406d0-512f-4b58-b143-f73ee05fbda1 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.290489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f52c16-7edf-454b-ad04-10ceb23672d5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.300683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00478b-579d-437b-a261-c9d7c860133a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.301868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00478b-579d-437b-a261-c9d7c860133a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.302467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00478b-579d-437b-a261-c9d7c860133a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.303111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00478b-579d-437b-a261-c9d7c860133a admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.305135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00478b-579d-437b-a261-c9d7c860133a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.309524 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00478b-579d-437b-a261-c9d7c860133a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.334068 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1009/2033] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:07.334320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc03b6-d6d4-44ad-b131-13837984049e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.334444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056118ad-2d51-49e2-bac5-ffa68a6e23ba None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.334444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056118ad-2d51-49e2-bac5-ffa68a6e23ba None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.334617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc03b6-d6d4-44ad-b131-13837984049e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.334721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056118ad-2d51-49e2-bac5-ffa68a6e23ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.334806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc03b6-d6d4-44ad-b131-13837984049e admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.334806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc03b6-d6d4-44ad-b131-13837984049e admin admin] PATH_INFO: `/v3/users/8e4225954fc443e0a0413fd3c4d3871e` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.337644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc03b6-d6d4-44ad-b131-13837984049e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e4225954fc443e0a0413fd3c4d3871e)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.342600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc03b6-d6d4-44ad-b131-13837984049e admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.357789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-056118ad-2d51-49e2-bac5-ffa68a6e23ba None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:07.364494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-056118ad-2d51-49e2-bac5-ffa68a6e23ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:07.371878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:07.373380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1010/2034] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:07.374411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056118ad-2d51-49e2-bac5-ffa68a6e23ba None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.386795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056118ad-2d51-49e2-bac5-ffa68a6e23ba None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.392646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048d5aa-aba6-418b-b574-f55e5ade2829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.393487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048d5aa-aba6-418b-b574-f55e5ade2829 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.393771 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048d5aa-aba6-418b-b574-f55e5ade2829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.394270 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048d5aa-aba6-418b-b574-f55e5ade2829 admin admin] PATH_INFO: `/v3/projects/6d25fc6cb4754e7b9b2da37091351ffb/users/5920648a631d4539a6cadfb255fd7a68/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.396560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048d5aa-aba6-418b-b574-f55e5ade2829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d25fc6cb4754e7b9b2da37091351ffb, user_id=5920648a631d4539a6cadfb255fd7a68, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.399626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048d5aa-aba6-418b-b574-f55e5ade2829 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.414708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1011/2035] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:07 2022] PUT /identity/v3/projects/6d25fc6cb4754e7b9b2da37091351ffb/users/5920648a631d4539a6cadfb255fd7a68/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:07.425892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048d5aa-aba6-418b-b574-f55e5ade2829 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.428000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3048d5aa-aba6-418b-b574-f55e5ade2829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:07.461252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-43cc03b6-d6d4-44ad-b131-13837984049e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e4225954fc443e0a0413fd3c4d3871e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:07.462815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:07.481628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-43cc03b6-d6d4-44ad-b131-13837984049e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e4225954fc443e0a0413fd3c4d3871e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:07.482704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-43cc03b6-d6d4-44ad-b131-13837984049e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e4225954fc443e0a0413fd3c4d3871e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:07.490964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-43cc03b6-d6d4-44ad-b131-13837984049e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e4225954fc443e0a0413fd3c4d3871e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:07.491840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-43cc03b6-d6d4-44ad-b131-13837984049e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e4225954fc443e0a0413fd3c4d3871e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:07.517838 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1025/2036] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:07 2022] DELETE /identity/v3/users/8e4225954fc443e0a0413fd3c4d3871e => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:07.520864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aa56d-1009-45ff-9daa-a096afdfcd18 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.521706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aa56d-1009-45ff-9daa-a096afdfcd18 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.522460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aa56d-1009-45ff-9daa-a096afdfcd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.534482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048d5aa-aba6-418b-b574-f55e5ade2829 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.583610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-313aa56d-1009-45ff-9daa-a096afdfcd18 None None] MFA Rules not processed for user `5920648a631d4539a6cadfb255fd7a68`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:07.585901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-313aa56d-1009-45ff-9daa-a096afdfcd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:07.662852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e30a8-e401-41dc-b329-cc6ce01e236b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.663970 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e30a8-e401-41dc-b329-cc6ce01e236b service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.664265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e30a8-e401-41dc-b329-cc6ce01e236b service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.664609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e30a8-e401-41dc-b329-cc6ce01e236b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.667889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-485e30a8-e401-41dc-b329-cc6ce01e236b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:07.711702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1026/2037] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:07.713866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc6cff-1ba7-4e62-b923-2816ba6109b3 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.714374 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc6cff-1ba7-4e62-b923-2816ba6109b3 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.714791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc6cff-1ba7-4e62-b923-2816ba6109b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.739021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-a2bc6cff-1ba7-4e62-b923-2816ba6109b3 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:07.740249 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a2bc6cff-1ba7-4e62-b923-2816ba6109b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:07.773508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e30a8-e401-41dc-b329-cc6ce01e236b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.775017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e30a8-e401-41dc-b329-cc6ce01e236b service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.787331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1012/2038] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:07.788242 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e30a8-e401-41dc-b329-cc6ce01e236b service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.790405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-485e30a8-e401-41dc-b329-cc6ce01e236b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:07.837511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1027/2039] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:07.849339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc6cff-1ba7-4e62-b923-2816ba6109b3 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.851386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a2bc6cff-1ba7-4e62-b923-2816ba6109b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:07.877155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc6cff-1ba7-4e62-b923-2816ba6109b3 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.882840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbe16b-aacc-4fa0-8a85-9c6f2f529a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.883692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbe16b-aacc-4fa0-8a85-9c6f2f529a41 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.884093 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbe16b-aacc-4fa0-8a85-9c6f2f529a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.884272 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e30a8-e401-41dc-b329-cc6ce01e236b service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.884636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbe16b-aacc-4fa0-8a85-9c6f2f529a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.891771 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f1076-834c-4177-8223-15a84332b996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.892607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f1076-834c-4177-8223-15a84332b996 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.893793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f1076-834c-4177-8223-15a84332b996 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.894086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f1076-834c-4177-8223-15a84332b996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.900589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbe16b-aacc-4fa0-8a85-9c6f2f529a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.903120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbe16b-aacc-4fa0-8a85-9c6f2f529a41 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.911082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f1076-834c-4177-8223-15a84332b996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.912890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f1076-834c-4177-8223-15a84332b996 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.928860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1013/2041] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:07.929055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1028/2041] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:08.056431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f1076-834c-4177-8223-15a84332b996 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.066411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f1076-834c-4177-8223-15a84332b996 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.072128 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d184a-6879-4313-a897-aadaa0dbffe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.073367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d184a-6879-4313-a897-aadaa0dbffe8 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.073810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d184a-6879-4313-a897-aadaa0dbffe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.074426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d184a-6879-4313-a897-aadaa0dbffe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.077070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a15d184a-6879-4313-a897-aadaa0dbffe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:08.121664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:08.171391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d184a-6879-4313-a897-aadaa0dbffe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.173057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d184a-6879-4313-a897-aadaa0dbffe8 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.187448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1014/2042] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:08.300540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbe16b-aacc-4fa0-8a85-9c6f2f529a41 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.303425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3ebbe16b-aacc-4fa0-8a85-9c6f2f529a41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:08.322933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:08.330719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbe16b-aacc-4fa0-8a85-9c6f2f529a41 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.350919 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d184a-6879-4313-a897-aadaa0dbffe8 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.353035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a15d184a-6879-4313-a897-aadaa0dbffe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:08.353704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6634d33-b628-4acf-a49b-5b50326a499c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.357013 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6634d33-b628-4acf-a49b-5b50326a499c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.357765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6634d33-b628-4acf-a49b-5b50326a499c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.358247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6634d33-b628-4acf-a49b-5b50326a499c admin admin] PATH_INFO: `/v3/projects/c2e526d5f3854195ac646a6532e4e43f` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.361217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6634d33-b628-4acf-a49b-5b50326a499c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e526d5f3854195ac646a6532e4e43f)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.365779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6634d33-b628-4acf-a49b-5b50326a499c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.368606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d184a-6879-4313-a897-aadaa0dbffe8 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.375859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.376789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.377089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.377363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] PATH_INFO: `/v3/projects/3e697ff997e04921ab46574cc830ab0f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.378640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e697ff997e04921ab46574cc830ab0f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.380649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.472997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e697ff997e04921ab46574cc830ab0f', 'request_id': 'req-23e79ef5-1629-4005-8fdc-2d2726693f79'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.481181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e697ff997e04921ab46574cc830ab0f', 'request_id': 'req-23e79ef5-1629-4005-8fdc-2d2726693f79'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.486503 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a6634d33-b628-4acf-a49b-5b50326a499c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e526d5f3854195ac646a6532e4e43f', 'request_id': 'req-a6634d33-b628-4acf-a49b-5b50326a499c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.490596 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e697ff997e04921ab46574cc830ab0f', 'request_id': 'req-23e79ef5-1629-4005-8fdc-2d2726693f79'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.495867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a6634d33-b628-4acf-a49b-5b50326a499c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e526d5f3854195ac646a6532e4e43f', 'request_id': 'req-a6634d33-b628-4acf-a49b-5b50326a499c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.502729 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a6634d33-b628-4acf-a49b-5b50326a499c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e526d5f3854195ac646a6532e4e43f', 'request_id': 'req-a6634d33-b628-4acf-a49b-5b50326a499c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.507081 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] The token cache is being invalidate because project 3e697ff997e04921ab46574cc830ab0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:08.507773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.508631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.511195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1015/2043] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:08 2022] DELETE /identity/v3/projects/3e697ff997e04921ab46574cc830ab0f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:08.516036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.516892 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a6634d33-b628-4acf-a49b-5b50326a499c admin admin] The token cache is being invalidate because project c2e526d5f3854195ac646a6532e4e43f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:08.517171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a6634d33-b628-4acf-a49b-5b50326a499c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.517863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a6634d33-b628-4acf-a49b-5b50326a499c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.518057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:08.519910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1029/2044] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:08 2022] DELETE /identity/v3/projects/c2e526d5f3854195ac646a6532e4e43f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:08.524850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579278c-0a76-46a8-bed6-c5bc005064d4 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.525129 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579278c-0a76-46a8-bed6-c5bc005064d4 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.526450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579278c-0a76-46a8-bed6-c5bc005064d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.546908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-5579278c-0a76-46a8-bed6-c5bc005064d4 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:08.548157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5579278c-0a76-46a8-bed6-c5bc005064d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:08.596010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e79ef5-1629-4005-8fdc-2d2726693f79 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.646665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1030/2045] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:08.656488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e34aec-4ff2-4122-aa7f-b79267f28402 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.656799 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e34aec-4ff2-4122-aa7f-b79267f28402 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.657087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e34aec-4ff2-4122-aa7f-b79267f28402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.676453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-80e34aec-4ff2-4122-aa7f-b79267f28402 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:08.677746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-80e34aec-4ff2-4122-aa7f-b79267f28402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:08.683930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:08.691555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.692933 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.693607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1031/2046] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:08.693795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.693931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] PATH_INFO: `/v3/users/838a69445d2041d8a994cea8fada2d66` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.695725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=838a69445d2041d8a994cea8fada2d66)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.698372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e34aec-4ff2-4122-aa7f-b79267f28402 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.698587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.708663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e34aec-4ff2-4122-aa7f-b79267f28402 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.714683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7caa7-ae4e-428c-8313-873b47b2fa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.715779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7caa7-ae4e-428c-8313-873b47b2fa2f admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.715779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7caa7-ae4e-428c-8313-873b47b2fa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.715966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7caa7-ae4e-428c-8313-873b47b2fa2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.716711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7caa7-ae4e-428c-8313-873b47b2fa2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:08.717023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7caa7-ae4e-428c-8313-873b47b2fa2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.718643 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7caa7-ae4e-428c-8313-873b47b2fa2f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.732020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1032/2047] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:08.736360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7caa7-ae4e-428c-8313-873b47b2fa2f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.749322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7caa7-ae4e-428c-8313-873b47b2fa2f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.755476 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975db8c-0c31-4c35-b81b-d5d8094909d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.756196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975db8c-0c31-4c35-b81b-d5d8094909d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.756597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975db8c-0c31-4c35-b81b-d5d8094909d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.757036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975db8c-0c31-4c35-b81b-d5d8094909d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.758282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975db8c-0c31-4c35-b81b-d5d8094909d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.760291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975db8c-0c31-4c35-b81b-d5d8094909d0 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.771575 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '838a69445d2041d8a994cea8fada2d66'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.785052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '838a69445d2041d8a994cea8fada2d66'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.800773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1033/2048] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:08 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:08.803280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975db8c-0c31-4c35-b81b-d5d8094909d0 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.803968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '838a69445d2041d8a994cea8fada2d66'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.814597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '838a69445d2041d8a994cea8fada2d66'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.816002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '838a69445d2041d8a994cea8fada2d66'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:08.825859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1016/2049] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:08 2022] DELETE /identity/v3/users/838a69445d2041d8a994cea8fada2d66 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:08.880267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:08.919178 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975db8c-0c31-4c35-b81b-d5d8094909d0 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.017461 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff134ff-fc24-4f11-827f-4535e11e0237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.018528 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff134ff-fc24-4f11-827f-4535e11e0237 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.018820 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff134ff-fc24-4f11-827f-4535e11e0237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.019107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff134ff-fc24-4f11-827f-4535e11e0237 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.019833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff134ff-fc24-4f11-827f-4535e11e0237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.021863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff134ff-fc24-4f11-827f-4535e11e0237 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.048340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1034/2050] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:08 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:09.052349 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.054271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:09.073982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002b1eb-ad0c-4064-b6de-8a8980598054 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.081440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb0ebd-73bb-4c25-bbce-33443f427431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.082521 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb0ebd-73bb-4c25-bbce-33443f427431 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.082876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb0ebd-73bb-4c25-bbce-33443f427431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.083165 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb0ebd-73bb-4c25-bbce-33443f427431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.083917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb0ebd-73bb-4c25-bbce-33443f427431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.086579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb0ebd-73bb-4c25-bbce-33443f427431 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.101427 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1017/2051] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:09.106515 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff134ff-fc24-4f11-827f-4535e11e0237 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.120890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff134ff-fc24-4f11-827f-4535e11e0237 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.127145 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c05f9-eb97-47c2-a719-06db1e542648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.127980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c05f9-eb97-47c2-a719-06db1e542648 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.128293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c05f9-eb97-47c2-a719-06db1e542648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.128601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c05f9-eb97-47c2-a719-06db1e542648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.129407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c05f9-eb97-47c2-a719-06db1e542648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.131360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c05f9-eb97-47c2-a719-06db1e542648 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.145481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1035/2052] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:09.151782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb0ebd-73bb-4c25-bbce-33443f427431 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.167410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb0ebd-73bb-4c25-bbce-33443f427431 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.175143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c05f9-eb97-47c2-a719-06db1e542648 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.175381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b27219-ea8e-4063-b942-27b5d933ad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.176865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b27219-ea8e-4063-b942-27b5d933ad4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.177458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b27219-ea8e-4063-b942-27b5d933ad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.178632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b27219-ea8e-4063-b942-27b5d933ad4c admin admin] PATH_INFO: `/v3/projects/a87b5950d61d43fa90e998caa0b7f72b/users/3afa9f73453e4bc58c0f0a32f9e70fe8/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.178713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-082c05f9-eb97-47c2-a719-06db1e542648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:09.206064 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c05f9-eb97-47c2-a719-06db1e542648 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.214944 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7182b7-323c-400c-9bf1-1a4de58e17f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.216643 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7182b7-323c-400c-9bf1-1a4de58e17f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.217162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7182b7-323c-400c-9bf1-1a4de58e17f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.217631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7182b7-323c-400c-9bf1-1a4de58e17f0 admin admin] PATH_INFO: `/v3/projects/7cbd9edebbd049719d4738b56a29a136` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.219702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7182b7-323c-400c-9bf1-1a4de58e17f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cbd9edebbd049719d4738b56a29a136)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.223257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7182b7-323c-400c-9bf1-1a4de58e17f0 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.224059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b27219-ea8e-4063-b942-27b5d933ad4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87b5950d61d43fa90e998caa0b7f72b, user_id=3afa9f73453e4bc58c0f0a32f9e70fe8, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.229202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b27219-ea8e-4063-b942-27b5d933ad4c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.252403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1018/2053] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:09 2022] PUT /identity/v3/projects/a87b5950d61d43fa90e998caa0b7f72b/users/3afa9f73453e4bc58c0f0a32f9e70fe8/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:09.271873 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b529a-ee08-41a0-a3cb-395ed9a6528e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.272148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b529a-ee08-41a0-a3cb-395ed9a6528e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.272430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b529a-ee08-41a0-a3cb-395ed9a6528e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.305919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ca7182b7-323c-400c-9bf1-1a4de58e17f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cbd9edebbd049719d4738b56a29a136', 'request_id': 'req-ca7182b7-323c-400c-9bf1-1a4de58e17f0'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:09.309877 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-060b529a-ee08-41a0-a3cb-395ed9a6528e None None] MFA Rules not processed for user `3afa9f73453e4bc58c0f0a32f9e70fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:09.311501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-060b529a-ee08-41a0-a3cb-395ed9a6528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:09.318972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ca7182b7-323c-400c-9bf1-1a4de58e17f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cbd9edebbd049719d4738b56a29a136', 'request_id': 'req-ca7182b7-323c-400c-9bf1-1a4de58e17f0'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:09.329031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ca7182b7-323c-400c-9bf1-1a4de58e17f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cbd9edebbd049719d4738b56a29a136', 'request_id': 'req-ca7182b7-323c-400c-9bf1-1a4de58e17f0'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:09.344336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ca7182b7-323c-400c-9bf1-1a4de58e17f0 admin admin] The token cache is being invalidate because project 7cbd9edebbd049719d4738b56a29a136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:09.350938 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ca7182b7-323c-400c-9bf1-1a4de58e17f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:09.352424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ca7182b7-323c-400c-9bf1-1a4de58e17f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:09.355194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1036/2054] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:09 2022] DELETE /identity/v3/projects/7cbd9edebbd049719d4738b56a29a136 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:09.360706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a71f0-6e51-4f96-91be-3b893026de44 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.361034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a71f0-6e51-4f96-91be-3b893026de44 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.361300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a71f0-6e51-4f96-91be-3b893026de44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.389751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ed2a71f0-6e51-4f96-91be-3b893026de44 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:09.391824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ed2a71f0-6e51-4f96-91be-3b893026de44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:09.407915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1019/2055] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:09.423358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b529a-ee08-41a0-a3cb-395ed9a6528e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.424643 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-060b529a-ee08-41a0-a3cb-395ed9a6528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:09.461825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:09.486037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1037/2056] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:09.493977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a38bd3-458d-414d-9fd7-2c8fc38f5953 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.494413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a38bd3-458d-414d-9fd7-2c8fc38f5953 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.494688 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a38bd3-458d-414d-9fd7-2c8fc38f5953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.494850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:09.516267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-99a38bd3-458d-414d-9fd7-2c8fc38f5953 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:09.517647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-99a38bd3-458d-414d-9fd7-2c8fc38f5953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:09.528677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1038/2057] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:09.539152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a38bd3-458d-414d-9fd7-2c8fc38f5953 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.550152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b529a-ee08-41a0-a3cb-395ed9a6528e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.556998 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0f7d4-9346-4cd6-ac88-ddd968ca02ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.557245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a38bd3-458d-414d-9fd7-2c8fc38f5953 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.558363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0f7d4-9346-4cd6-ac88-ddd968ca02ab service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.558852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0f7d4-9346-4cd6-ac88-ddd968ca02ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.559404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0f7d4-9346-4cd6-ac88-ddd968ca02ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.566365 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b5319-ce86-4a61-bebb-4162a2b82f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.567674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b5319-ce86-4a61-bebb-4162a2b82f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.568273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b5319-ce86-4a61-bebb-4162a2b82f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.568864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b5319-ce86-4a61-bebb-4162a2b82f90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.570358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b5319-ce86-4a61-bebb-4162a2b82f90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:09.570986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b5319-ce86-4a61-bebb-4162a2b82f90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.574635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b5319-ce86-4a61-bebb-4162a2b82f90 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.577393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0f7d4-9346-4cd6-ac88-ddd968ca02ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.579023 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0f7d4-9346-4cd6-ac88-ddd968ca02ab service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.595583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1020/2058] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:09.601678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1039/2059] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:09.604318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0f7d4-9346-4cd6-ac88-ddd968ca02ab service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.618011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0f7d4-9346-4cd6-ac88-ddd968ca02ab service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.625194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13817aba-7353-4bb0-a72f-440afd7800c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.625889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13817aba-7353-4bb0-a72f-440afd7800c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.626182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13817aba-7353-4bb0-a72f-440afd7800c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.626457 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13817aba-7353-4bb0-a72f-440afd7800c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.627194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13817aba-7353-4bb0-a72f-440afd7800c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.629183 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13817aba-7353-4bb0-a72f-440afd7800c7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.670863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1021/2060] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:09 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:09.675862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b5319-ce86-4a61-bebb-4162a2b82f90 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.686842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b5319-ce86-4a61-bebb-4162a2b82f90 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.770155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96da9-3e03-436f-b285-3bffd5fa45de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.771190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96da9-3e03-436f-b285-3bffd5fa45de admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.771472 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96da9-3e03-436f-b285-3bffd5fa45de admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.771752 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96da9-3e03-436f-b285-3bffd5fa45de admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.772494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a96da9-3e03-436f-b285-3bffd5fa45de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.774937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a96da9-3e03-436f-b285-3bffd5fa45de admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.802228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1040/2061] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:09 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:09.808170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13817aba-7353-4bb0-a72f-440afd7800c7 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.821334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13817aba-7353-4bb0-a72f-440afd7800c7 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.829728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8b13c-da64-4ca3-8bff-5ecf7787c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.830771 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8b13c-da64-4ca3-8bff-5ecf7787c225 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.831131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8b13c-da64-4ca3-8bff-5ecf7787c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.831678 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8b13c-da64-4ca3-8bff-5ecf7787c225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.832909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8b13c-da64-4ca3-8bff-5ecf7787c225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.835119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8b13c-da64-4ca3-8bff-5ecf7787c225 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.857427 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1022/2062] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:09.864186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96da9-3e03-436f-b285-3bffd5fa45de admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.876504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96da9-3e03-436f-b285-3bffd5fa45de admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.882762 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d9d22-42f1-46bd-8a56-68893437e8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.883527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d9d22-42f1-46bd-8a56-68893437e8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.883811 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d9d22-42f1-46bd-8a56-68893437e8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.884094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d9d22-42f1-46bd-8a56-68893437e8e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.885449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d9d22-42f1-46bd-8a56-68893437e8e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.887140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d9d22-42f1-46bd-8a56-68893437e8e1 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.898822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1041/2063] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:09.903972 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8b13c-da64-4ca3-8bff-5ecf7787c225 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.914987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8b13c-da64-4ca3-8bff-5ecf7787c225 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.923117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cf223-aa88-421b-8d21-38560635e6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.924066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cf223-aa88-421b-8d21-38560635e6c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.924379 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cf223-aa88-421b-8d21-38560635e6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.924841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cf223-aa88-421b-8d21-38560635e6c5 admin admin] PATH_INFO: `/v3/projects/1de2da2eb4944bf78b3cc97ea215efd4/users/ecfa654797784f27b43a28d917f33916/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.943109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cf223-aa88-421b-8d21-38560635e6c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de2da2eb4944bf78b3cc97ea215efd4, user_id=ecfa654797784f27b43a28d917f33916, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.946046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cf223-aa88-421b-8d21-38560635e6c5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.975401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1023/2064] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:09 2022] PUT /identity/v3/projects/1de2da2eb4944bf78b3cc97ea215efd4/users/ecfa654797784f27b43a28d917f33916/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:09.987322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da328e-dbef-4707-b6f0-233c3c81466c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.987620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da328e-dbef-4707-b6f0-233c3c81466c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.987934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da328e-dbef-4707-b6f0-233c3c81466c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.025628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-62da328e-dbef-4707-b6f0-233c3c81466c None None] MFA Rules not processed for user `ecfa654797784f27b43a28d917f33916`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:10.027504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-62da328e-dbef-4707-b6f0-233c3c81466c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:10.161295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1042/2065] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:10.168157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cf223-aa88-421b-8d21-38560635e6c5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.261285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cf223-aa88-421b-8d21-38560635e6c5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.269944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838765ef-257d-4607-80c3-221543ffa7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.271174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838765ef-257d-4607-80c3-221543ffa7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.271695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838765ef-257d-4607-80c3-221543ffa7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.272219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838765ef-257d-4607-80c3-221543ffa7b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.273766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838765ef-257d-4607-80c3-221543ffa7b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.277741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838765ef-257d-4607-80c3-221543ffa7b4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.321904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1024/2066] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:10 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:10.329450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da328e-dbef-4707-b6f0-233c3c81466c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.350026 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da328e-dbef-4707-b6f0-233c3c81466c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.408501 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:10.433754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7274a-dc4a-4e7e-9007-89614cbadcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.434459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7274a-dc4a-4e7e-9007-89614cbadcab admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.434749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7274a-dc4a-4e7e-9007-89614cbadcab admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.435032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7274a-dc4a-4e7e-9007-89614cbadcab admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.435756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7274a-dc4a-4e7e-9007-89614cbadcab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.437766 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7274a-dc4a-4e7e-9007-89614cbadcab admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.465031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1043/2067] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:10 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:10.469544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838765ef-257d-4607-80c3-221543ffa7b4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.481806 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838765ef-257d-4607-80c3-221543ffa7b4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.486717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea760c-fdb5-4316-85e6-eb598e858c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.487399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea760c-fdb5-4316-85e6-eb598e858c22 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.487681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea760c-fdb5-4316-85e6-eb598e858c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.487954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea760c-fdb5-4316-85e6-eb598e858c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.488737 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea760c-fdb5-4316-85e6-eb598e858c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.490389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea760c-fdb5-4316-85e6-eb598e858c22 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.502597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1025/2068] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:10.508016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7274a-dc4a-4e7e-9007-89614cbadcab admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.527983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7274a-dc4a-4e7e-9007-89614cbadcab admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.536209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf4666-7fc7-4772-86e4-2fed4b716c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.537474 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf4666-7fc7-4772-86e4-2fed4b716c18 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.537980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf4666-7fc7-4772-86e4-2fed4b716c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.538489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf4666-7fc7-4772-86e4-2fed4b716c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.539872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf4666-7fc7-4772-86e4-2fed4b716c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.543731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf4666-7fc7-4772-86e4-2fed4b716c18 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.559005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1044/2069] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:10.563595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea760c-fdb5-4316-85e6-eb598e858c22 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.577235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea760c-fdb5-4316-85e6-eb598e858c22 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.591851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1bb64-aa0b-44e8-a275-1511e968ddcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.591851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1bb64-aa0b-44e8-a275-1511e968ddcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.591851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1bb64-aa0b-44e8-a275-1511e968ddcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.591851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1bb64-aa0b-44e8-a275-1511e968ddcb admin admin] PATH_INFO: `/v3/projects/4377e03834384210b72dfd606c57b690/users/f4b49d0ecea44e61bb9eccf5490a0ac2/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.618626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1bb64-aa0b-44e8-a275-1511e968ddcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4377e03834384210b72dfd606c57b690, user_id=f4b49d0ecea44e61bb9eccf5490a0ac2, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.624137 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1bb64-aa0b-44e8-a275-1511e968ddcb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.646141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1026/2070] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:10 2022] PUT /identity/v3/projects/4377e03834384210b72dfd606c57b690/users/f4b49d0ecea44e61bb9eccf5490a0ac2/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:10.660019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98357cad-0c36-4005-ae01-4f8e251b5bd3 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.660466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98357cad-0c36-4005-ae01-4f8e251b5bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.660864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98357cad-0c36-4005-ae01-4f8e251b5bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.696912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-98357cad-0c36-4005-ae01-4f8e251b5bd3 None None] MFA Rules not processed for user `f4b49d0ecea44e61bb9eccf5490a0ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:10.698783 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-98357cad-0c36-4005-ae01-4f8e251b5bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:10.808844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1045/2071] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:10.814281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1bb64-aa0b-44e8-a275-1511e968ddcb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.860646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:10.918256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1bb64-aa0b-44e8-a275-1511e968ddcb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.918256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b7860-20ea-43d7-aa1d-876449d2e9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.918256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b7860-20ea-43d7-aa1d-876449d2e9c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.918256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b7860-20ea-43d7-aa1d-876449d2e9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.918878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b7860-20ea-43d7-aa1d-876449d2e9c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.918878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b7860-20ea-43d7-aa1d-876449d2e9c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.920946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b7860-20ea-43d7-aa1d-876449d2e9c7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.966313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1027/2072] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:10 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:10.967608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98357cad-0c36-4005-ae01-4f8e251b5bd3 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.982673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98357cad-0c36-4005-ae01-4f8e251b5bd3 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.086723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729261b-97a4-4122-8f5d-b4422c6bf95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.088635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729261b-97a4-4122-8f5d-b4422c6bf95f admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.089344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729261b-97a4-4122-8f5d-b4422c6bf95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.089934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729261b-97a4-4122-8f5d-b4422c6bf95f admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.091539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729261b-97a4-4122-8f5d-b4422c6bf95f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.099117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729261b-97a4-4122-8f5d-b4422c6bf95f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.158365 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1046/2073] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:10 2022] POST /identity/v3/users => generated 299 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:11.164101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b7860-20ea-43d7-aa1d-876449d2e9c7 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.176899 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b7860-20ea-43d7-aa1d-876449d2e9c7 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.183004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b424f5-7287-4417-9a9e-d8f104a70d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.183700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b424f5-7287-4417-9a9e-d8f104a70d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.184548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b424f5-7287-4417-9a9e-d8f104a70d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.184844 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b424f5-7287-4417-9a9e-d8f104a70d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.185665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b424f5-7287-4417-9a9e-d8f104a70d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.187785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b424f5-7287-4417-9a9e-d8f104a70d68 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.201683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1028/2074] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:11.208056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729261b-97a4-4122-8f5d-b4422c6bf95f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.220340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729261b-97a4-4122-8f5d-b4422c6bf95f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.226448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6cc82-9e38-48cd-88fd-be2779eb62bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.227450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6cc82-9e38-48cd-88fd-be2779eb62bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.227747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6cc82-9e38-48cd-88fd-be2779eb62bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.228021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6cc82-9e38-48cd-88fd-be2779eb62bd admin admin] PATH_INFO: `/v3/projects/a5c24d3205d2434a9b09e9368d9fa965/users/b441530e4b24471db57d48d6a4a6fc94/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.252291 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6cc82-9e38-48cd-88fd-be2779eb62bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5c24d3205d2434a9b09e9368d9fa965, user_id=b441530e4b24471db57d48d6a4a6fc94, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.256292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6cc82-9e38-48cd-88fd-be2779eb62bd admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.288707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1047/2075] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:11 2022] PUT /identity/v3/projects/a5c24d3205d2434a9b09e9368d9fa965/users/b441530e4b24471db57d48d6a4a6fc94/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:11.290654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b424f5-7287-4417-9a9e-d8f104a70d68 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.315546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b424f5-7287-4417-9a9e-d8f104a70d68 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.441607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7ea1d-364b-4c38-aabd-24a692b35335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.442725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ea1d-364b-4c38-aabd-24a692b35335 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.443201 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ea1d-364b-4c38-aabd-24a692b35335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.443550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ea1d-364b-4c38-aabd-24a692b35335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.444800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7ea1d-364b-4c38-aabd-24a692b35335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.446776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7ea1d-364b-4c38-aabd-24a692b35335 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.462595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1029/2076] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:11.470263 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6cc82-9e38-48cd-88fd-be2779eb62bd admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.488244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6cc82-9e38-48cd-88fd-be2779eb62bd admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.494198 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df4aa0-f43c-49bb-8878-180113570d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.495135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df4aa0-f43c-49bb-8878-180113570d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.495449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df4aa0-f43c-49bb-8878-180113570d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.495722 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df4aa0-f43c-49bb-8878-180113570d0d admin admin] PATH_INFO: `/v3/projects/a5c24d3205d2434a9b09e9368d9fa965/users/b441530e4b24471db57d48d6a4a6fc94/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.497874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df4aa0-f43c-49bb-8878-180113570d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5c24d3205d2434a9b09e9368d9fa965, user_id=b441530e4b24471db57d48d6a4a6fc94, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.500639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df4aa0-f43c-49bb-8878-180113570d0d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.514575 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1048/2077] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:11 2022] PUT /identity/v3/projects/a5c24d3205d2434a9b09e9368d9fa965/users/b441530e4b24471db57d48d6a4a6fc94/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:11.526472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a66b8d-8237-464d-bf69-ec052def8a60 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.527099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a66b8d-8237-464d-bf69-ec052def8a60 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.527236 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a66b8d-8237-464d-bf69-ec052def8a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.565584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-01a66b8d-8237-464d-bf69-ec052def8a60 None None] MFA Rules not processed for user `b441530e4b24471db57d48d6a4a6fc94`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:11.566903 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-01a66b8d-8237-464d-bf69-ec052def8a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:11.689412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1030/2078] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:11.709482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df4aa0-f43c-49bb-8878-180113570d0d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.723714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:11.810494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df4aa0-f43c-49bb-8878-180113570d0d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.814448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e927d9-968b-45d4-a7ae-218aad05faf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.815129 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e927d9-968b-45d4-a7ae-218aad05faf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.815412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e927d9-968b-45d4-a7ae-218aad05faf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.815687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e927d9-968b-45d4-a7ae-218aad05faf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.888312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e927d9-968b-45d4-a7ae-218aad05faf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.890279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e927d9-968b-45d4-a7ae-218aad05faf2 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.902400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1049/2079] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:12.662653 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a66b8d-8237-464d-bf69-ec052def8a60 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:12.664124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-01a66b8d-8237-464d-bf69-ec052def8a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:12.706018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:12.764123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a66b8d-8237-464d-bf69-ec052def8a60 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:12.773278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:12.774559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:12.775553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:12.776038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] PATH_INFO: `/v3/users/758ab03d602043dc9d8c87bf61671db8` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:12.777917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=758ab03d602043dc9d8c87bf61671db8)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:12.781323 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:12.855958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '758ab03d602043dc9d8c87bf61671db8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:12.864083 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '758ab03d602043dc9d8c87bf61671db8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:12.883664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '758ab03d602043dc9d8c87bf61671db8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:12.898588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '758ab03d602043dc9d8c87bf61671db8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:12.899268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '758ab03d602043dc9d8c87bf61671db8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:12.908357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1031/2080] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:12 2022] DELETE /identity/v3/users/758ab03d602043dc9d8c87bf61671db8 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:12.912919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a3b05-4324-4439-b443-daee5c703320 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:12.913832 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a3b05-4324-4439-b443-daee5c703320 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:12.914318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a3b05-4324-4439-b443-daee5c703320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:12.934466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-794a3b05-4324-4439-b443-daee5c703320 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:12.935755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-794a3b05-4324-4439-b443-daee5c703320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:13.029458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1050/2081] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:13.042204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:13.044468 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:13.110002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725a150-1ef4-4180-bb80-7b64f600b535 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:13.113851 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0b323-100e-4710-8abb-495343ac6329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:13.114524 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0b323-100e-4710-8abb-495343ac6329 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:13.114804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0b323-100e-4710-8abb-495343ac6329 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:13.115082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0b323-100e-4710-8abb-495343ac6329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:13.127882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0b323-100e-4710-8abb-495343ac6329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:13.128249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0b323-100e-4710-8abb-495343ac6329 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:13.139671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1032/2082] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:13.544136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a3b05-4324-4439-b443-daee5c703320 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:13.548303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-794a3b05-4324-4439-b443-daee5c703320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:13.573864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a3b05-4324-4439-b443-daee5c703320 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:13.581940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:13.584508 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:13.585122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:13.585796 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] PATH_INFO: `/v3/projects/56e7ba4044754f438856fb9c7c5e5b3a` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:13.589480 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] RBAC: Authorizing `identity:delete_project(project_id=56e7ba4044754f438856fb9c7c5e5b3a)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:13.593801 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:13.698441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56e7ba4044754f438856fb9c7c5e5b3a', 'request_id': 'req-58ef02b2-42db-435f-90bc-bf407d7c448f'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:13.704845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:13.722641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56e7ba4044754f438856fb9c7c5e5b3a', 'request_id': 'req-58ef02b2-42db-435f-90bc-bf407d7c448f'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:13.724844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56e7ba4044754f438856fb9c7c5e5b3a', 'request_id': 'req-58ef02b2-42db-435f-90bc-bf407d7c448f'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:13.748359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] The token cache is being invalidate because project 56e7ba4044754f438856fb9c7c5e5b3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:13.748708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:13.749567 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:13.751481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1051/2083] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:13 2022] DELETE /identity/v3/projects/56e7ba4044754f438856fb9c7c5e5b3a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:13.755739 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0b323-100e-4710-8abb-495343ac6329 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:13.768837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0b323-100e-4710-8abb-495343ac6329 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:13.814543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:13.817030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:13.838130 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:13.877437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:13.878993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:13.879514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:13.880021 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] PATH_INFO: `/v3/users/5920648a631d4539a6cadfb255fd7a68` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:13.882607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5920648a631d4539a6cadfb255fd7a68)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:13.886874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:13.923363 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef02b2-42db-435f-90bc-bf407d7c448f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:13.928328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9ea42-61f5-46e7-87b7-c8035f281568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:13.929130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9ea42-61f5-46e7-87b7-c8035f281568 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:13.929601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9ea42-61f5-46e7-87b7-c8035f281568 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:13.929822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9ea42-61f5-46e7-87b7-c8035f281568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:13.933771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-71c9ea42-61f5-46e7-87b7-c8035f281568 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:13.971794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5920648a631d4539a6cadfb255fd7a68'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:13.983094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5920648a631d4539a6cadfb255fd7a68'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.002254 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5920648a631d4539a6cadfb255fd7a68'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.013899 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5920648a631d4539a6cadfb255fd7a68'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.015319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5920648a631d4539a6cadfb255fd7a68'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.023815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1033/2084] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:13 2022] DELETE /identity/v3/users/5920648a631d4539a6cadfb255fd7a68 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:14.025992 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9ea42-61f5-46e7-87b7-c8035f281568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:14.028785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9ea42-61f5-46e7-87b7-c8035f281568 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:14.057332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1052/2085] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:14.532098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:14.534536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:14.639163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872155cc-110f-430f-ad2b-a4f848699248 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:14.645060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef89b7-21e4-4453-b842-ffb99f72a209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:14.646334 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef89b7-21e4-4453-b842-ffb99f72a209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:14.646722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef89b7-21e4-4453-b842-ffb99f72a209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:14.647074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef89b7-21e4-4453-b842-ffb99f72a209 admin admin] PATH_INFO: `/v3/projects/6d25fc6cb4754e7b9b2da37091351ffb` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:14.648711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef89b7-21e4-4453-b842-ffb99f72a209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d25fc6cb4754e7b9b2da37091351ffb)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:14.650942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef89b7-21e4-4453-b842-ffb99f72a209 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:14.724431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f6ef89b7-21e4-4453-b842-ffb99f72a209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d25fc6cb4754e7b9b2da37091351ffb', 'request_id': 'req-f6ef89b7-21e4-4453-b842-ffb99f72a209'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.731915 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9ea42-61f5-46e7-87b7-c8035f281568 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:14.733502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-71c9ea42-61f5-46e7-87b7-c8035f281568 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:14.736001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f6ef89b7-21e4-4453-b842-ffb99f72a209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d25fc6cb4754e7b9b2da37091351ffb', 'request_id': 'req-f6ef89b7-21e4-4453-b842-ffb99f72a209'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.749397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f6ef89b7-21e4-4453-b842-ffb99f72a209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d25fc6cb4754e7b9b2da37091351ffb', 'request_id': 'req-f6ef89b7-21e4-4453-b842-ffb99f72a209'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.751903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9ea42-61f5-46e7-87b7-c8035f281568 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:14.752139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:14.771720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f6ef89b7-21e4-4453-b842-ffb99f72a209 admin admin] The token cache is being invalidate because project 6d25fc6cb4754e7b9b2da37091351ffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:14.772668 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f6ef89b7-21e4-4453-b842-ffb99f72a209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.772668 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f6ef89b7-21e4-4453-b842-ffb99f72a209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.780594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1034/2086] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:14 2022] DELETE /identity/v3/projects/6d25fc6cb4754e7b9b2da37091351ffb => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:14.786170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d09ea3-35ec-457d-a9b2-b49c18716510 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:14.786722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d09ea3-35ec-457d-a9b2-b49c18716510 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:14.787207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d09ea3-35ec-457d-a9b2-b49c18716510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:14.821602 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-30d09ea3-35ec-457d-a9b2-b49c18716510 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:14.823380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-30d09ea3-35ec-457d-a9b2-b49c18716510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:14.823695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984d7ff-7611-49ba-b8fa-16569649b8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:14.824635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984d7ff-7611-49ba-b8fa-16569649b8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:14.824924 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984d7ff-7611-49ba-b8fa-16569649b8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:14.825207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984d7ff-7611-49ba-b8fa-16569649b8c6 admin admin] PATH_INFO: `/v3/users/3afa9f73453e4bc58c0f0a32f9e70fe8` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:14.826574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984d7ff-7611-49ba-b8fa-16569649b8c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3afa9f73453e4bc58c0f0a32f9e70fe8)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:14.828645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984d7ff-7611-49ba-b8fa-16569649b8c6 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:14.840296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1035/2087] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:14.840296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d09ea3-35ec-457d-a9b2-b49c18716510 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:14.840296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-30d09ea3-35ec-457d-a9b2-b49c18716510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:14.895826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:DELETE=8,keystone:SELECT=92 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:14.914862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-3984d7ff-7611-49ba-b8fa-16569649b8c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3afa9f73453e4bc58c0f0a32f9e70fe8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.926253 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-3984d7ff-7611-49ba-b8fa-16569649b8c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3afa9f73453e4bc58c0f0a32f9e70fe8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.927872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-3984d7ff-7611-49ba-b8fa-16569649b8c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3afa9f73453e4bc58c0f0a32f9e70fe8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.933578 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d09ea3-35ec-457d-a9b2-b49c18716510 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:14.937394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-3984d7ff-7611-49ba-b8fa-16569649b8c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3afa9f73453e4bc58c0f0a32f9e70fe8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.937695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad3025-56fd-45b5-84ea-c9e261c6d3b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:14.938383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-3984d7ff-7611-49ba-b8fa-16569649b8c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3afa9f73453e4bc58c0f0a32f9e70fe8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:14.938506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad3025-56fd-45b5-84ea-c9e261c6d3b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:14.938724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad3025-56fd-45b5-84ea-c9e261c6d3b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:14.939253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad3025-56fd-45b5-84ea-c9e261c6d3b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:14.941693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-74ad3025-56fd-45b5-84ea-c9e261c6d3b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:14.965009 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1053/2088] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:14 2022] DELETE /identity/v3/users/3afa9f73453e4bc58c0f0a32f9e70fe8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:14.966681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a8a7d-1fb2-48ad-b305-08dd06f41853 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:14.967120 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a8a7d-1fb2-48ad-b305-08dd06f41853 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:14.967546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a8a7d-1fb2-48ad-b305-08dd06f41853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:14.991493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-703a8a7d-1fb2-48ad-b305-08dd06f41853 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:14.992868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-703a8a7d-1fb2-48ad-b305-08dd06f41853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:15.040054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad3025-56fd-45b5-84ea-c9e261c6d3b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.047651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad3025-56fd-45b5-84ea-c9e261c6d3b6 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.055105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1036/2089] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:15.057098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f167ac-8dbe-4866-b1f1-6baa4a5da2e0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.057416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f167ac-8dbe-4866-b1f1-6baa4a5da2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.057720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f167ac-8dbe-4866-b1f1-6baa4a5da2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.083073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e2f167ac-8dbe-4866-b1f1-6baa4a5da2e0 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:15.084956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e2f167ac-8dbe-4866-b1f1-6baa4a5da2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:15.126779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1054/2090] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:15.131050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a8a7d-1fb2-48ad-b305-08dd06f41853 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.142975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a8a7d-1fb2-48ad-b305-08dd06f41853 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.149962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53711647-96d9-4ee5-a778-5ff64060c65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.150916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53711647-96d9-4ee5-a778-5ff64060c65f admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.151294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53711647-96d9-4ee5-a778-5ff64060c65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.151643 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53711647-96d9-4ee5-a778-5ff64060c65f admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.152522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53711647-96d9-4ee5-a778-5ff64060c65f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:15.152942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53711647-96d9-4ee5-a778-5ff64060c65f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.155030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53711647-96d9-4ee5-a778-5ff64060c65f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.171075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1055/2091] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:15.177063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53711647-96d9-4ee5-a778-5ff64060c65f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.192121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53711647-96d9-4ee5-a778-5ff64060c65f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.209827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ce8a9-c01e-4c08-9e57-150664da1d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.210793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ce8a9-c01e-4c08-9e57-150664da1d37 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.211623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ce8a9-c01e-4c08-9e57-150664da1d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.212118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ce8a9-c01e-4c08-9e57-150664da1d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.214411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ce8a9-c01e-4c08-9e57-150664da1d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.216723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ce8a9-c01e-4c08-9e57-150664da1d37 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.232119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1037/2092] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:15.252879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f167ac-8dbe-4866-b1f1-6baa4a5da2e0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.260934 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e2f167ac-8dbe-4866-b1f1-6baa4a5da2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:15.311231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1056/2093] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:15 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:15.316247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ce8a9-c01e-4c08-9e57-150664da1d37 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.359343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:15.367721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f167ac-8dbe-4866-b1f1-6baa4a5da2e0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.467436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb1e21-fe9e-4d35-aec8-d7bd1f4fc911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.468640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb1e21-fe9e-4d35-aec8-d7bd1f4fc911 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.469205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb1e21-fe9e-4d35-aec8-d7bd1f4fc911 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.469695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb1e21-fe9e-4d35-aec8-d7bd1f4fc911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.533100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ce8a9-c01e-4c08-9e57-150664da1d37 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.551188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7601dd-d62a-46b3-b761-4cbde5819ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.552386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7601dd-d62a-46b3-b761-4cbde5819ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.552933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7601dd-d62a-46b3-b761-4cbde5819ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.553423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7601dd-d62a-46b3-b761-4cbde5819ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.555467 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7601dd-d62a-46b3-b761-4cbde5819ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.559140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7601dd-d62a-46b3-b761-4cbde5819ae6 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.608747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1057/2094] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:15 2022] POST /identity/v3/users => generated 285 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:15.610687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7601dd-d62a-46b3-b761-4cbde5819ae6 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.612803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ab7601dd-d62a-46b3-b761-4cbde5819ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:15.631748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb1e21-fe9e-4d35-aec8-d7bd1f4fc911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.635397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7601dd-d62a-46b3-b761-4cbde5819ae6 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.637362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb1e21-fe9e-4d35-aec8-d7bd1f4fc911 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.659157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683b429-8ffc-4f34-9b19-00fe94280acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.659157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683b429-8ffc-4f34-9b19-00fe94280acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.659651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683b429-8ffc-4f34-9b19-00fe94280acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.659651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683b429-8ffc-4f34-9b19-00fe94280acd admin admin] PATH_INFO: `/v3/users/27e58ecac94c4133817d957f77355445` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.661051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683b429-8ffc-4f34-9b19-00fe94280acd admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e58ecac94c4133817d957f77355445)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.662062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1038/2095] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:15.662666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb1e21-fe9e-4d35-aec8-d7bd1f4fc911 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.662994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683b429-8ffc-4f34-9b19-00fe94280acd admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.666529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:15.675874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb1e21-fe9e-4d35-aec8-d7bd1f4fc911 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.682569 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ad98a-259c-4261-82ec-ad15f8705a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.683641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ad98a-259c-4261-82ec-ad15f8705a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.684470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ad98a-259c-4261-82ec-ad15f8705a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.684675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ad98a-259c-4261-82ec-ad15f8705a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.685764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ad98a-259c-4261-82ec-ad15f8705a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.687190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ad98a-259c-4261-82ec-ad15f8705a38 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.699883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1039/2096] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:15.706631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ad98a-259c-4261-82ec-ad15f8705a38 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.723437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ad98a-259c-4261-82ec-ad15f8705a38 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.733326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85547258-59a2-4a72-84de-5ccb61369121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.734030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85547258-59a2-4a72-84de-5ccb61369121 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.734502 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85547258-59a2-4a72-84de-5ccb61369121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.734816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85547258-59a2-4a72-84de-5ccb61369121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.736018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85547258-59a2-4a72-84de-5ccb61369121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.738682 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85547258-59a2-4a72-84de-5ccb61369121 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.738911 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-2683b429-8ffc-4f34-9b19-00fe94280acd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e58ecac94c4133817d957f77355445'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:15.758687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1040/2097] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:15.758961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-2683b429-8ffc-4f34-9b19-00fe94280acd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e58ecac94c4133817d957f77355445'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:15.758961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-2683b429-8ffc-4f34-9b19-00fe94280acd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e58ecac94c4133817d957f77355445'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:15.761981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85547258-59a2-4a72-84de-5ccb61369121 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.765960 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-85547258-59a2-4a72-84de-5ccb61369121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:15.774545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-2683b429-8ffc-4f34-9b19-00fe94280acd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e58ecac94c4133817d957f77355445'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:15.778169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-2683b429-8ffc-4f34-9b19-00fe94280acd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e58ecac94c4133817d957f77355445'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:15.795940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85547258-59a2-4a72-84de-5ccb61369121 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.807890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b02370-415c-44a1-b784-45b107013f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.811978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b02370-415c-44a1-b784-45b107013f2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.812626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b02370-415c-44a1-b784-45b107013f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.814187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b02370-415c-44a1-b784-45b107013f2c admin admin] PATH_INFO: `/v3/projects/62849039c2504feab54537898697ee61/users/6c234ceb8057478a88ef4ca0367afca6/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.818802 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1058/2098] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:15 2022] DELETE /identity/v3/users/27e58ecac94c4133817d957f77355445 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:15.825312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800626e-1984-4d92-b38c-d7976086bbb8 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.826389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800626e-1984-4d92-b38c-d7976086bbb8 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.828394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800626e-1984-4d92-b38c-d7976086bbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.848389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b02370-415c-44a1-b784-45b107013f2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=62849039c2504feab54537898697ee61, user_id=6c234ceb8057478a88ef4ca0367afca6, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.854723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b02370-415c-44a1-b784-45b107013f2c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.865850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-0800626e-1984-4d92-b38c-d7976086bbb8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:15.868238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0800626e-1984-4d92-b38c-d7976086bbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:15.891925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1041/2099] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:15 2022] PUT /identity/v3/projects/62849039c2504feab54537898697ee61/users/6c234ceb8057478a88ef4ca0367afca6/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:15.932232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5db53-74d2-4850-9832-88e71779736a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.933864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5db53-74d2-4850-9832-88e71779736a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.934564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5db53-74d2-4850-9832-88e71779736a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.976521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1059/2100] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:15.990492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800626e-1984-4d92-b38c-d7976086bbb8 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.995281 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0800626e-1984-4d92-b38c-d7976086bbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:16.000357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-84b5db53-74d2-4850-9832-88e71779736a None None] MFA Rules not processed for user `6c234ceb8057478a88ef4ca0367afca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:16.001675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-84b5db53-74d2-4850-9832-88e71779736a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:16.104073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:16.141631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1042/2101] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:16.146223 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5db53-74d2-4850-9832-88e71779736a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.150986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-84b5db53-74d2-4850-9832-88e71779736a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:16.167481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800626e-1984-4d92-b38c-d7976086bbb8 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.173993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ce471-ca72-40d2-ba72-75e15ce1f3cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.175109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce471-ca72-40d2-ba72-75e15ce1f3cd service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.175628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce471-ca72-40d2-ba72-75e15ce1f3cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.176090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce471-ca72-40d2-ba72-75e15ce1f3cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.185511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5db53-74d2-4850-9832-88e71779736a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.192120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0294aa7-7585-48ce-9982-3a81e3c29cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.193575 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ce471-ca72-40d2-ba72-75e15ce1f3cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.193915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0294aa7-7585-48ce-9982-3a81e3c29cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.194444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0294aa7-7585-48ce-9982-3a81e3c29cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.194959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0294aa7-7585-48ce-9982-3a81e3c29cf2 admin admin] PATH_INFO: `/v3/projects/a87b5950d61d43fa90e998caa0b7f72b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.197323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ce471-ca72-40d2-ba72-75e15ce1f3cd service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.198143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0294aa7-7585-48ce-9982-3a81e3c29cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a87b5950d61d43fa90e998caa0b7f72b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.202170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0294aa7-7585-48ce-9982-3a81e3c29cf2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.215893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1060/2102] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:16.217492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ce471-ca72-40d2-ba72-75e15ce1f3cd service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.245571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ce471-ca72-40d2-ba72-75e15ce1f3cd service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.253989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f08dec-1cc1-4f16-a5a4-3dddfa9ced43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.257028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f08dec-1cc1-4f16-a5a4-3dddfa9ced43 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.258210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f08dec-1cc1-4f16-a5a4-3dddfa9ced43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.262270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f08dec-1cc1-4f16-a5a4-3dddfa9ced43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.266388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f08dec-1cc1-4f16-a5a4-3dddfa9ced43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.289441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f08dec-1cc1-4f16-a5a4-3dddfa9ced43 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.321143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a0294aa7-7585-48ce-9982-3a81e3c29cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a87b5950d61d43fa90e998caa0b7f72b', 'request_id': 'req-a0294aa7-7585-48ce-9982-3a81e3c29cf2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:16.335043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a0294aa7-7585-48ce-9982-3a81e3c29cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a87b5950d61d43fa90e998caa0b7f72b', 'request_id': 'req-a0294aa7-7585-48ce-9982-3a81e3c29cf2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:16.351377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a0294aa7-7585-48ce-9982-3a81e3c29cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a87b5950d61d43fa90e998caa0b7f72b', 'request_id': 'req-a0294aa7-7585-48ce-9982-3a81e3c29cf2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:16.361192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1061/2103] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:16 2022] POST /identity/v3/projects => generated 369 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:16.361192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f08dec-1cc1-4f16-a5a4-3dddfa9ced43 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.371020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:16.371339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a0294aa7-7585-48ce-9982-3a81e3c29cf2 admin admin] The token cache is being invalidate because project a87b5950d61d43fa90e998caa0b7f72b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:16.371595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a0294aa7-7585-48ce-9982-3a81e3c29cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:16.372336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a0294aa7-7585-48ce-9982-3a81e3c29cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:16.375276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1043/2104] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:16 2022] DELETE /identity/v3/projects/a87b5950d61d43fa90e998caa0b7f72b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:16.386415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f82248-f3ca-4d8e-8499-8fde2aaf6056 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.386853 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f82248-f3ca-4d8e-8499-8fde2aaf6056 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.387280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f82248-f3ca-4d8e-8499-8fde2aaf6056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.424783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-37f82248-f3ca-4d8e-8499-8fde2aaf6056 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:16.426211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-37f82248-f3ca-4d8e-8499-8fde2aaf6056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:16.565965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f08dec-1cc1-4f16-a5a4-3dddfa9ced43 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.582192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10932cd-ee77-4e77-a314-d3657e898934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.583030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10932cd-ee77-4e77-a314-d3657e898934 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.583316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10932cd-ee77-4e77-a314-d3657e898934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.583597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10932cd-ee77-4e77-a314-d3657e898934 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.584554 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10932cd-ee77-4e77-a314-d3657e898934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.586357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10932cd-ee77-4e77-a314-d3657e898934 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.631617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1062/2105] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:16 2022] POST /identity/v3/users => generated 291 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:16.638270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10932cd-ee77-4e77-a314-d3657e898934 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.640687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b10932cd-ee77-4e77-a314-d3657e898934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:16.664837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10932cd-ee77-4e77-a314-d3657e898934 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.689223 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a549a7-6f94-4979-968e-70ea6eb4c1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.690166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a549a7-6f94-4979-968e-70ea6eb4c1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.690369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a549a7-6f94-4979-968e-70ea6eb4c1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.690718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a549a7-6f94-4979-968e-70ea6eb4c1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.691420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a549a7-6f94-4979-968e-70ea6eb4c1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.693210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a549a7-6f94-4979-968e-70ea6eb4c1a2 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.713294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1044/2106] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:16.716147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a5c06-ff74-40bc-a321-ba65617e80e5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.716853 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1063/2107] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:16.716973 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a5c06-ff74-40bc-a321-ba65617e80e5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.725413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a5c06-ff74-40bc-a321-ba65617e80e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.735257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a549a7-6f94-4979-968e-70ea6eb4c1a2 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.770062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7a4a5c06-ff74-40bc-a321-ba65617e80e5 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:16.770632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a549a7-6f94-4979-968e-70ea6eb4c1a2 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.772942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7a4a5c06-ff74-40bc-a321-ba65617e80e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:16.794395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0ca39-229d-47ea-8d39-c2dd3bb4c992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.795329 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0ca39-229d-47ea-8d39-c2dd3bb4c992 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.795621 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0ca39-229d-47ea-8d39-c2dd3bb4c992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.795903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0ca39-229d-47ea-8d39-c2dd3bb4c992 admin admin] PATH_INFO: `/v3/projects/43dc58493f0443b1a7d657f3d005b929/users/a7008ea98f3e487caeec8efd96082160/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.801854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1045/2108] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:16.812952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a5c06-ff74-40bc-a321-ba65617e80e5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.834301 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a5c06-ff74-40bc-a321-ba65617e80e5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.844550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0ca39-229d-47ea-8d39-c2dd3bb4c992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43dc58493f0443b1a7d657f3d005b929, user_id=a7008ea98f3e487caeec8efd96082160, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.847419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0ca39-229d-47ea-8d39-c2dd3bb4c992 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.856319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054211a-0eb7-47e9-b297-85dcba46e04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.857109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054211a-0eb7-47e9-b297-85dcba46e04c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.857423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054211a-0eb7-47e9-b297-85dcba46e04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.857719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054211a-0eb7-47e9-b297-85dcba46e04c admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.858531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054211a-0eb7-47e9-b297-85dcba46e04c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:16.858931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054211a-0eb7-47e9-b297-85dcba46e04c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.860643 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054211a-0eb7-47e9-b297-85dcba46e04c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.886642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1064/2109] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:16 2022] PUT /identity/v3/projects/43dc58493f0443b1a7d657f3d005b929/users/a7008ea98f3e487caeec8efd96082160/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:16.894285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1046/2110] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:16.899578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0ca39-229d-47ea-8d39-c2dd3bb4c992 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.900692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054211a-0eb7-47e9-b297-85dcba46e04c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.916922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054211a-0eb7-47e9-b297-85dcba46e04c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.051584 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0ca39-229d-47ea-8d39-c2dd3bb4c992 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.054302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:17.059477 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea283d5-b76c-42a5-a1c2-22672ce23444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.060352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea283d5-b76c-42a5-a1c2-22672ce23444 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.060783 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea283d5-b76c-42a5-a1c2-22672ce23444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.061221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea283d5-b76c-42a5-a1c2-22672ce23444 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.062361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea283d5-b76c-42a5-a1c2-22672ce23444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.064274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea283d5-b76c-42a5-a1c2-22672ce23444 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.079267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1065/2111] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:17.086761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea283d5-b76c-42a5-a1c2-22672ce23444 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.095538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad9b1e-ab2b-41e2-814c-14ca3b3b0ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.096321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad9b1e-ab2b-41e2-814c-14ca3b3b0ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.096735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad9b1e-ab2b-41e2-814c-14ca3b3b0ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.097084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad9b1e-ab2b-41e2-814c-14ca3b3b0ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.100722 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea283d5-b76c-42a5-a1c2-22672ce23444 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.107459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad9b1e-ab2b-41e2-814c-14ca3b3b0ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.109469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76f8df-a3dd-4147-8c2b-1b3bb0cd5062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.109617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad9b1e-ab2b-41e2-814c-14ca3b3b0ee2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.110665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76f8df-a3dd-4147-8c2b-1b3bb0cd5062 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.111142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76f8df-a3dd-4147-8c2b-1b3bb0cd5062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.111590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76f8df-a3dd-4147-8c2b-1b3bb0cd5062 admin admin] PATH_INFO: `/v3/projects/43dc58493f0443b1a7d657f3d005b929/users/a7008ea98f3e487caeec8efd96082160/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.114532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76f8df-a3dd-4147-8c2b-1b3bb0cd5062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43dc58493f0443b1a7d657f3d005b929, user_id=a7008ea98f3e487caeec8efd96082160, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.137330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76f8df-a3dd-4147-8c2b-1b3bb0cd5062 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.159807 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1047/2112] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:16 2022] POST /identity/v3/projects => generated 399 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:17.161091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad9b1e-ab2b-41e2-814c-14ca3b3b0ee2 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.163748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bfad9b1e-ab2b-41e2-814c-14ca3b3b0ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:17.164623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1066/2113] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:17 2022] PUT /identity/v3/projects/43dc58493f0443b1a7d657f3d005b929/users/a7008ea98f3e487caeec8efd96082160/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:17.165717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76f8df-a3dd-4147-8c2b-1b3bb0cd5062 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.234802 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:17.263914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad9b1e-ab2b-41e2-814c-14ca3b3b0ee2 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.265451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76f8df-a3dd-4147-8c2b-1b3bb0cd5062 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.271120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.271581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b21f53-cb3b-40bc-a5ac-5b68d9c29f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.272019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.272300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.272677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] PATH_INFO: `/v3/projects/45d539964a964635b8ebe7afb291f1d1` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.273118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b21f53-cb3b-40bc-a5ac-5b68d9c29f7b admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.273379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b21f53-cb3b-40bc-a5ac-5b68d9c29f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.273631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b21f53-cb3b-40bc-a5ac-5b68d9c29f7b admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.275223 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d539964a964635b8ebe7afb291f1d1)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.275347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b21f53-cb3b-40bc-a5ac-5b68d9c29f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.277407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.277836 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b21f53-cb3b-40bc-a5ac-5b68d9c29f7b admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.302693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1067/2114] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:17 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:17.304971 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aea0a-ea52-4882-96b7-714e7ca59e9e None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.305492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aea0a-ea52-4882-96b7-714e7ca59e9e None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.307164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98aea0a-ea52-4882-96b7-714e7ca59e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.336446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d539964a964635b8ebe7afb291f1d1', 'request_id': 'req-aae1754f-92a8-41ec-97e6-76d9645f6abd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:17.346694 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d539964a964635b8ebe7afb291f1d1', 'request_id': 'req-aae1754f-92a8-41ec-97e6-76d9645f6abd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:17.349022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f98aea0a-ea52-4882-96b7-714e7ca59e9e None None] MFA Rules not processed for user `a7008ea98f3e487caeec8efd96082160`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:17.350289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f98aea0a-ea52-4882-96b7-714e7ca59e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:17.355998 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d539964a964635b8ebe7afb291f1d1', 'request_id': 'req-aae1754f-92a8-41ec-97e6-76d9645f6abd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:17.372955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] The token cache is being invalidate because project 45d539964a964635b8ebe7afb291f1d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:17.373355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:17.374199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:17.381293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1048/2115] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:17 2022] DELETE /identity/v3/projects/45d539964a964635b8ebe7afb291f1d1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:17.381293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.383617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:17.472282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1068/2116] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:17 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:17.473262 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98aea0a-ea52-4882-96b7-714e7ca59e9e None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.474556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f98aea0a-ea52-4882-96b7-714e7ca59e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:17.489329 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1754f-92a8-41ec-97e6-76d9645f6abd admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.490868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98aea0a-ea52-4882-96b7-714e7ca59e9e None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.495203 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517060f-692d-4963-91c6-0dbcac78d47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.495888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517060f-692d-4963-91c6-0dbcac78d47e admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.496175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517060f-692d-4963-91c6-0dbcac78d47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.496455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517060f-692d-4963-91c6-0dbcac78d47e admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.496733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.497227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517060f-692d-4963-91c6-0dbcac78d47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.497774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.498060 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.498446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] PATH_INFO: `/v3/users/2fa864c0a75046c1b9096584413e56b0` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.498862 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517060f-692d-4963-91c6-0dbcac78d47e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.500166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fa864c0a75046c1b9096584413e56b0)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.502790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.512227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1049/2117] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:17.513138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517060f-692d-4963-91c6-0dbcac78d47e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.515000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f517060f-692d-4963-91c6-0dbcac78d47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:17.521863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:17.575641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fa864c0a75046c1b9096584413e56b0'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:17.589520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fa864c0a75046c1b9096584413e56b0'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:17.590266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fa864c0a75046c1b9096584413e56b0'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:17.604111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fa864c0a75046c1b9096584413e56b0'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:17.604848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fa864c0a75046c1b9096584413e56b0'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:17.612430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517060f-692d-4963-91c6-0dbcac78d47e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.616399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85534d-4c39-4bc5-adf8-cecdf85a59ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.616997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85534d-4c39-4bc5-adf8-cecdf85a59ad service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.617280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85534d-4c39-4bc5-adf8-cecdf85a59ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.617555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85534d-4c39-4bc5-adf8-cecdf85a59ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.620227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9b85534d-4c39-4bc5-adf8-cecdf85a59ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:17.633235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85534d-4c39-4bc5-adf8-cecdf85a59ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.634774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85534d-4c39-4bc5-adf8-cecdf85a59ad service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.638542 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1069/2118] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:17 2022] DELETE /identity/v3/users/2fa864c0a75046c1b9096584413e56b0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:17.640027 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.648468 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1050/2119] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:17 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:17.649676 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:17.715116 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:17.719844 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85534d-4c39-4bc5-adf8-cecdf85a59ad service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.721621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9b85534d-4c39-4bc5-adf8-cecdf85a59ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:17.758055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85534d-4c39-4bc5-adf8-cecdf85a59ad service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.784420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecae1cb-ec4b-4334-b568-40b7100a226d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.792260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfabedb-0e5b-4ca2-bb91-fc1f08c3acda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.793032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfabedb-0e5b-4ca2-bb91-fc1f08c3acda admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.793386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfabedb-0e5b-4ca2-bb91-fc1f08c3acda admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.793648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfabedb-0e5b-4ca2-bb91-fc1f08c3acda admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.794830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfabedb-0e5b-4ca2-bb91-fc1f08c3acda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.796873 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfabedb-0e5b-4ca2-bb91-fc1f08c3acda admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.813220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1070/2120] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:17.818078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfabedb-0e5b-4ca2-bb91-fc1f08c3acda admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.834197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfabedb-0e5b-4ca2-bb91-fc1f08c3acda admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.841502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17782e19-3407-47e6-9a69-350da413f2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.842956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17782e19-3407-47e6-9a69-350da413f2aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.843462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17782e19-3407-47e6-9a69-350da413f2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.843939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17782e19-3407-47e6-9a69-350da413f2aa admin admin] PATH_INFO: `/v3/projects/2ffb60a72ee543deb455efffff800417/users/f2a73df58810444697a41f8e851d8f0d/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.853995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524ae9d-345d-40f2-9ddf-0eb31cc1421a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.855025 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524ae9d-345d-40f2-9ddf-0eb31cc1421a service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.855422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524ae9d-345d-40f2-9ddf-0eb31cc1421a service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.855957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524ae9d-345d-40f2-9ddf-0eb31cc1421a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.860009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c524ae9d-345d-40f2-9ddf-0eb31cc1421a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:17.871538 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17782e19-3407-47e6-9a69-350da413f2aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffb60a72ee543deb455efffff800417, user_id=f2a73df58810444697a41f8e851d8f0d, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.875275 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17782e19-3407-47e6-9a69-350da413f2aa admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.938784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1071/2121] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:17 2022] PUT /identity/v3/projects/2ffb60a72ee543deb455efffff800417/users/f2a73df58810444697a41f8e851d8f0d/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:17.940456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da4382-49d3-4de8-8f4f-1fd7e5308989 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.940816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da4382-49d3-4de8-8f4f-1fd7e5308989 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.941094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da4382-49d3-4de8-8f4f-1fd7e5308989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.954206 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524ae9d-345d-40f2-9ddf-0eb31cc1421a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.955846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524ae9d-345d-40f2-9ddf-0eb31cc1421a service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.971279 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1051/2122] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:17 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:18.016244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f0da4382-49d3-4de8-8f4f-1fd7e5308989 None None] MFA Rules not processed for user `f2a73df58810444697a41f8e851d8f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:18.017726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f0da4382-49d3-4de8-8f4f-1fd7e5308989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:18.195845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1072/2123] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:18.252418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524ae9d-345d-40f2-9ddf-0eb31cc1421a service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:18.254478 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c524ae9d-345d-40f2-9ddf-0eb31cc1421a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:18.279813 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524ae9d-345d-40f2-9ddf-0eb31cc1421a service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:18.404808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e331d8-f231-4b75-835e-7574fe1d5a6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:18.407983 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e331d8-f231-4b75-835e-7574fe1d5a6a service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:18.408611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e331d8-f231-4b75-835e-7574fe1d5a6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:18.409354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e331d8-f231-4b75-835e-7574fe1d5a6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:18.434111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e331d8-f231-4b75-835e-7574fe1d5a6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:18.437139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e331d8-f231-4b75-835e-7574fe1d5a6a service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:18.473264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1052/2124] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:18.495694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da4382-49d3-4de8-8f4f-1fd7e5308989 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:18.497390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f0da4382-49d3-4de8-8f4f-1fd7e5308989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:18.683195 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da4382-49d3-4de8-8f4f-1fd7e5308989 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:18.690506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a90b9-b610-4789-a9d4-19680ba713fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:18.692532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a90b9-b610-4789-a9d4-19680ba713fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:18.692532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a90b9-b610-4789-a9d4-19680ba713fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:18.692918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a90b9-b610-4789-a9d4-19680ba713fe admin admin] PATH_INFO: `/v3/projects/28cf8b7948544bfc9e9a690c69da8937` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:18.694943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a90b9-b610-4789-a9d4-19680ba713fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=28cf8b7948544bfc9e9a690c69da8937)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:18.698207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a90b9-b610-4789-a9d4-19680ba713fe admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:18.761122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:18.787877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-754a90b9-b610-4789-a9d4-19680ba713fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28cf8b7948544bfc9e9a690c69da8937', 'request_id': 'req-754a90b9-b610-4789-a9d4-19680ba713fe'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:18.799566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-754a90b9-b610-4789-a9d4-19680ba713fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28cf8b7948544bfc9e9a690c69da8937', 'request_id': 'req-754a90b9-b610-4789-a9d4-19680ba713fe'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:18.822285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-754a90b9-b610-4789-a9d4-19680ba713fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28cf8b7948544bfc9e9a690c69da8937', 'request_id': 'req-754a90b9-b610-4789-a9d4-19680ba713fe'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:18.839397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-754a90b9-b610-4789-a9d4-19680ba713fe admin admin] The token cache is being invalidate because project 28cf8b7948544bfc9e9a690c69da8937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:18.839858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-754a90b9-b610-4789-a9d4-19680ba713fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:18.840928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-754a90b9-b610-4789-a9d4-19680ba713fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:18.844019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1073/2125] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:18 2022] DELETE /identity/v3/projects/28cf8b7948544bfc9e9a690c69da8937 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:18.851188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c93168-503c-4a1c-bc3f-0f98c9973f3f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:18.851609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c93168-503c-4a1c-bc3f-0f98c9973f3f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:18.851978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c93168-503c-4a1c-bc3f-0f98c9973f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:18.863611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:18.883624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-54c93168-503c-4a1c-bc3f-0f98c9973f3f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:18.885989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-54c93168-503c-4a1c-bc3f-0f98c9973f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:19.028171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1053/2126] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:19.030310 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a5286-2c29-4685-aa12-26bc0af953d0 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.030717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a5286-2c29-4685-aa12-26bc0af953d0 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.031161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a5286-2c29-4685-aa12-26bc0af953d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.063845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-3b7a5286-2c29-4685-aa12-26bc0af953d0 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:19.066422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3b7a5286-2c29-4685-aa12-26bc0af953d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:19.087992 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1074/2127] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:19.094616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c93168-503c-4a1c-bc3f-0f98c9973f3f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.121558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c93168-503c-4a1c-bc3f-0f98c9973f3f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.135009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead32c3-1c1a-4957-bd8a-9e0c2a0d944a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.136604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead32c3-1c1a-4957-bd8a-9e0c2a0d944a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.137395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead32c3-1c1a-4957-bd8a-9e0c2a0d944a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.138123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ead32c3-1c1a-4957-bd8a-9e0c2a0d944a admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.139760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead32c3-1c1a-4957-bd8a-9e0c2a0d944a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:19.140559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead32c3-1c1a-4957-bd8a-9e0c2a0d944a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.143812 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ead32c3-1c1a-4957-bd8a-9e0c2a0d944a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:19.172692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1054/2128] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:19.179332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a5286-2c29-4685-aa12-26bc0af953d0 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.207163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a5286-2c29-4685-aa12-26bc0af953d0 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.224968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53af5ff-fd7b-4102-9186-bb579acb6880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.226719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53af5ff-fd7b-4102-9186-bb579acb6880 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.228620 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53af5ff-fd7b-4102-9186-bb579acb6880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.231126 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53af5ff-fd7b-4102-9186-bb579acb6880 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.238327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53af5ff-fd7b-4102-9186-bb579acb6880 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.240396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53af5ff-fd7b-4102-9186-bb579acb6880 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:19.300061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1075/2129] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:19 2022] POST /identity/v3/projects => generated 391 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:19.305587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead32c3-1c1a-4957-bd8a-9e0c2a0d944a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.320904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ead32c3-1c1a-4957-bd8a-9e0c2a0d944a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.434449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce9109-3691-4827-b2b9-67a75e7bf9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.435472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce9109-3691-4827-b2b9-67a75e7bf9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.435893 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce9109-3691-4827-b2b9-67a75e7bf9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.436315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce9109-3691-4827-b2b9-67a75e7bf9f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.437752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce9109-3691-4827-b2b9-67a75e7bf9f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.441934 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce9109-3691-4827-b2b9-67a75e7bf9f9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:19.472743 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1055/2130] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:19 2022] POST /identity/v3/users => generated 296 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:19.474983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53af5ff-fd7b-4102-9186-bb579acb6880 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.491816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53af5ff-fd7b-4102-9186-bb579acb6880 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.499430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca561c01-383a-431b-99ec-f690431d4e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.500282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca561c01-383a-431b-99ec-f690431d4e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.500724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca561c01-383a-431b-99ec-f690431d4e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.501034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca561c01-383a-431b-99ec-f690431d4e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.501858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca561c01-383a-431b-99ec-f690431d4e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.503562 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca561c01-383a-431b-99ec-f690431d4e48 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:19.518269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1076/2131] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:19.527864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce9109-3691-4827-b2b9-67a75e7bf9f9 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.563391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce9109-3691-4827-b2b9-67a75e7bf9f9 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.580232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75ffdf-de24-4706-9abb-a64742c1dc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.580874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ffdf-de24-4706-9abb-a64742c1dc4d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.581169 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ffdf-de24-4706-9abb-a64742c1dc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.581409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75ffdf-de24-4706-9abb-a64742c1dc4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.582253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75ffdf-de24-4706-9abb-a64742c1dc4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.584026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75ffdf-de24-4706-9abb-a64742c1dc4d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:19.625064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1056/2132] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:19.630877 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca561c01-383a-431b-99ec-f690431d4e48 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.659071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca561c01-383a-431b-99ec-f690431d4e48 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.670823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b55fbf-6bae-484c-8fbd-87aecac550a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.672343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b55fbf-6bae-484c-8fbd-87aecac550a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.673088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b55fbf-6bae-484c-8fbd-87aecac550a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.673602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b55fbf-6bae-484c-8fbd-87aecac550a4 admin admin] PATH_INFO: `/v3/projects/156f20101bf144d4accc3610faca358e/users/a0e13c7e9e3442dd923e790c6c6259ba/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.703531 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b55fbf-6bae-484c-8fbd-87aecac550a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=156f20101bf144d4accc3610faca358e, user_id=a0e13c7e9e3442dd923e790c6c6259ba, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.709214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b55fbf-6bae-484c-8fbd-87aecac550a4 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:19.736218 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1077/2133] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:19 2022] PUT /identity/v3/projects/156f20101bf144d4accc3610faca358e/users/a0e13c7e9e3442dd923e790c6c6259ba/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:19.748905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e4467-e3cc-4596-9b51-9655411a7f0c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.749809 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e4467-e3cc-4596-9b51-9655411a7f0c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.750747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e4467-e3cc-4596-9b51-9655411a7f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.831917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-889e4467-e3cc-4596-9b51-9655411a7f0c None None] MFA Rules not processed for user `a0e13c7e9e3442dd923e790c6c6259ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:19.835189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-889e4467-e3cc-4596-9b51-9655411a7f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:19.983530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:19.998006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1057/2134] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:20.002871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b55fbf-6bae-484c-8fbd-87aecac550a4 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.128100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b55fbf-6bae-484c-8fbd-87aecac550a4 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.135808 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e19c78-dee9-42f8-83af-2c2dc81cc065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.137149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e19c78-dee9-42f8-83af-2c2dc81cc065 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.137633 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e19c78-dee9-42f8-83af-2c2dc81cc065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.138071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e19c78-dee9-42f8-83af-2c2dc81cc065 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.139202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e19c78-dee9-42f8-83af-2c2dc81cc065 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.142930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e19c78-dee9-42f8-83af-2c2dc81cc065 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.199038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1078/2135] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:19 2022] POST /identity/v3/projects => generated 393 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:20.203603 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e4467-e3cc-4596-9b51-9655411a7f0c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.226824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e4467-e3cc-4596-9b51-9655411a7f0c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.256098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e19c78-dee9-42f8-83af-2c2dc81cc065 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.259166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b9e19c78-dee9-42f8-83af-2c2dc81cc065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:20.368968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88c05b-abe3-4045-aa00-978498a785dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.370177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88c05b-abe3-4045-aa00-978498a785dc admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.370442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88c05b-abe3-4045-aa00-978498a785dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.370970 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88c05b-abe3-4045-aa00-978498a785dc admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.372070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88c05b-abe3-4045-aa00-978498a785dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.376427 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88c05b-abe3-4045-aa00-978498a785dc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.377229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:20.426310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1058/2136] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:20 2022] POST /identity/v3/users => generated 303 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:20.433027 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e19c78-dee9-42f8-83af-2c2dc81cc065 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.434289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88c05b-abe3-4045-aa00-978498a785dc admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.449710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88c05b-abe3-4045-aa00-978498a785dc admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.450237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.451058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.451690 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.452146 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] PATH_INFO: `/v3/users/ecfa654797784f27b43a28d917f33916` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.454006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecfa654797784f27b43a28d917f33916)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.456367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.456743 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e805a-9f67-41eb-8004-c3b033460d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.457596 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e805a-9f67-41eb-8004-c3b033460d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.457977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e805a-9f67-41eb-8004-c3b033460d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.458391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e805a-9f67-41eb-8004-c3b033460d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.459372 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e805a-9f67-41eb-8004-c3b033460d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.461475 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e805a-9f67-41eb-8004-c3b033460d18 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.480767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1059/2137] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:20.482877 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e805a-9f67-41eb-8004-c3b033460d18 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.487689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-be9e805a-9f67-41eb-8004-c3b033460d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:20.516891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e805a-9f67-41eb-8004-c3b033460d18 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.523658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e4f3d-ce3f-4705-9f1e-9129346a7151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.524633 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e4f3d-ce3f-4705-9f1e-9129346a7151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.524934 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e4f3d-ce3f-4705-9f1e-9129346a7151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.525331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e4f3d-ce3f-4705-9f1e-9129346a7151 admin admin] PATH_INFO: `/v3/users/7c6730fc49df438083b5da8f0c331f3f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.526687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e4f3d-ce3f-4705-9f1e-9129346a7151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c6730fc49df438083b5da8f0c331f3f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.528710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e4f3d-ce3f-4705-9f1e-9129346a7151 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.539972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecfa654797784f27b43a28d917f33916'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:20.550682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecfa654797784f27b43a28d917f33916'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:20.551466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecfa654797784f27b43a28d917f33916'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:20.561776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecfa654797784f27b43a28d917f33916'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:20.563301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecfa654797784f27b43a28d917f33916'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:20.595377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d71e4f3d-ce3f-4705-9f1e-9129346a7151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c6730fc49df438083b5da8f0c331f3f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:20.597820 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1079/2138] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:20 2022] DELETE /identity/v3/users/ecfa654797784f27b43a28d917f33916 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:20.602459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.602459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:20.610805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d71e4f3d-ce3f-4705-9f1e-9129346a7151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c6730fc49df438083b5da8f0c331f3f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:20.633091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d71e4f3d-ce3f-4705-9f1e-9129346a7151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c6730fc49df438083b5da8f0c331f3f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:20.649509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d71e4f3d-ce3f-4705-9f1e-9129346a7151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c6730fc49df438083b5da8f0c331f3f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:20.651805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d71e4f3d-ce3f-4705-9f1e-9129346a7151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c6730fc49df438083b5da8f0c331f3f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:20.663104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1060/2139] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:20 2022] DELETE /identity/v3/users/7c6730fc49df438083b5da8f0c331f3f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:20.666377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97459564-51c4-4ca2-9965-efb792e18a7c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.666377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97459564-51c4-4ca2-9965-efb792e18a7c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.666652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97459564-51c4-4ca2-9965-efb792e18a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.699111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-97459564-51c4-4ca2-9965-efb792e18a7c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:20.701765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-97459564-51c4-4ca2-9965-efb792e18a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:20.724392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b4235-1b5b-4308-82b5-c9e480867686 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.787586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:20.816561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1061/2140] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:20.818979 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebe2cb-9376-4f3d-9841-644449b13aed None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.819486 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebe2cb-9376-4f3d-9841-644449b13aed None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.819965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebe2cb-9376-4f3d-9841-644449b13aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.838646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e66833-bc83-4fa1-9650-e2a7edde16ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.840109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e66833-bc83-4fa1-9650-e2a7edde16ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.840833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e66833-bc83-4fa1-9650-e2a7edde16ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.841296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e66833-bc83-4fa1-9650-e2a7edde16ba admin admin] PATH_INFO: `/v3/projects/a22190f157ba424d8ae39d7b5705fabe/users/2f9263206d1749f8ab39b5999689ae35/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.855231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-70ebe2cb-9376-4f3d-9841-644449b13aed None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:20.857458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-70ebe2cb-9376-4f3d-9841-644449b13aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:20.866511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:20.868015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1062/2141] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:20.869822 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebe2cb-9376-4f3d-9841-644449b13aed None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.871193 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-70ebe2cb-9376-4f3d-9841-644449b13aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:20.882699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e66833-bc83-4fa1-9650-e2a7edde16ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22190f157ba424d8ae39d7b5705fabe, user_id=2f9263206d1749f8ab39b5999689ae35, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.889911 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e66833-bc83-4fa1-9650-e2a7edde16ba admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.924916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1080/2142] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:20 2022] PUT /identity/v3/projects/a22190f157ba424d8ae39d7b5705fabe/users/2f9263206d1749f8ab39b5999689ae35/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:20.926794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e66833-bc83-4fa1-9650-e2a7edde16ba admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.933670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-39e66833-bc83-4fa1-9650-e2a7edde16ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:21.049442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e66833-bc83-4fa1-9650-e2a7edde16ba admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.059482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c332a37-cf34-4d43-a609-d32e4e82f03f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.061359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c332a37-cf34-4d43-a609-d32e4e82f03f service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.061796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebe2cb-9376-4f3d-9841-644449b13aed None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.062490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c332a37-cf34-4d43-a609-d32e4e82f03f service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.063633 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c332a37-cf34-4d43-a609-d32e4e82f03f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.074997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f71f9d-01d9-4587-af15-4cf9c49bf506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.076533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f71f9d-01d9-4587-af15-4cf9c49bf506 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.082977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f71f9d-01d9-4587-af15-4cf9c49bf506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.082977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f71f9d-01d9-4587-af15-4cf9c49bf506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.207052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f71f9d-01d9-4587-af15-4cf9c49bf506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.209497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f71f9d-01d9-4587-af15-4cf9c49bf506 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.216604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c332a37-cf34-4d43-a609-d32e4e82f03f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.220055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c332a37-cf34-4d43-a609-d32e4e82f03f service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.226328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1063/2143] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:21.227736 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f71f9d-01d9-4587-af15-4cf9c49bf506 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.230812 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-28f71f9d-01d9-4587-af15-4cf9c49bf506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:21.242604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1081/2144] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:21.250582 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f71f9d-01d9-4587-af15-4cf9c49bf506 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.257803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c5ea3-c2c5-43ff-ac2d-ede6c26b0161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.257803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c5ea3-c2c5-43ff-ac2d-ede6c26b0161 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.258204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c5ea3-c2c5-43ff-ac2d-ede6c26b0161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.258597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c5ea3-c2c5-43ff-ac2d-ede6c26b0161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.260079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c5ea3-c2c5-43ff-ac2d-ede6c26b0161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.262964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c5ea3-c2c5-43ff-ac2d-ede6c26b0161 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.275710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1064/2145] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:21.280151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c332a37-cf34-4d43-a609-d32e4e82f03f service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.292220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c332a37-cf34-4d43-a609-d32e4e82f03f service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.297560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e68bef-da1e-4601-ac7a-5be20c87d9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.300277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e68bef-da1e-4601-ac7a-5be20c87d9c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.300277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e68bef-da1e-4601-ac7a-5be20c87d9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.300277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e68bef-da1e-4601-ac7a-5be20c87d9c8 admin admin] PATH_INFO: `/v3/projects/a22190f157ba424d8ae39d7b5705fabe/users/2f9263206d1749f8ab39b5999689ae35/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.302676 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e68bef-da1e-4601-ac7a-5be20c87d9c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22190f157ba424d8ae39d7b5705fabe, user_id=2f9263206d1749f8ab39b5999689ae35, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.305060 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e68bef-da1e-4601-ac7a-5be20c87d9c8 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.328626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1082/2146] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:21 2022] PUT /identity/v3/projects/a22190f157ba424d8ae39d7b5705fabe/users/2f9263206d1749f8ab39b5999689ae35/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:21.332663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77b2b6-d0ce-4bc9-a232-12835aec801c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.333096 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77b2b6-d0ce-4bc9-a232-12835aec801c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.333283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77b2b6-d0ce-4bc9-a232-12835aec801c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.373854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6f77b2b6-d0ce-4bc9-a232-12835aec801c None None] MFA Rules not processed for user `2f9263206d1749f8ab39b5999689ae35`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:21.375238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6f77b2b6-d0ce-4bc9-a232-12835aec801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:21.515411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1065/2147] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:21.541833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e68bef-da1e-4601-ac7a-5be20c87d9c8 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.547294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-e7e68bef-da1e-4601-ac7a-5be20c87d9c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:21.710646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e68bef-da1e-4601-ac7a-5be20c87d9c8 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.720372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb138b6e-455f-47d3-b643-de71e1b67d21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.721927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138b6e-455f-47d3-b643-de71e1b67d21 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.722561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138b6e-455f-47d3-b643-de71e1b67d21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.723174 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138b6e-455f-47d3-b643-de71e1b67d21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.761656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb138b6e-455f-47d3-b643-de71e1b67d21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.765453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb138b6e-455f-47d3-b643-de71e1b67d21 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.795331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1083/2148] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:21.863876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77b2b6-d0ce-4bc9-a232-12835aec801c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.875726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77b2b6-d0ce-4bc9-a232-12835aec801c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.880379 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c186f-7401-492d-80cc-f006f3cdb74b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.881545 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c186f-7401-492d-80cc-f006f3cdb74b service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.882058 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c186f-7401-492d-80cc-f006f3cdb74b service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.883550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c186f-7401-492d-80cc-f006f3cdb74b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.886324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-554c186f-7401-492d-80cc-f006f3cdb74b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:21.893451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:22.006951 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c186f-7401-492d-80cc-f006f3cdb74b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:22.011520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c186f-7401-492d-80cc-f006f3cdb74b service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:22.041201 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1066/2149] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:22.223718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb138b6e-455f-47d3-b643-de71e1b67d21 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:22.225814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bb138b6e-455f-47d3-b643-de71e1b67d21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:22.337920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:22.454868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb138b6e-455f-47d3-b643-de71e1b67d21 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:22.461333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eec2ac-846f-4e73-9218-049dabcbbdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:22.462504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eec2ac-846f-4e73-9218-049dabcbbdb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:22.463016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eec2ac-846f-4e73-9218-049dabcbbdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:22.463337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eec2ac-846f-4e73-9218-049dabcbbdb8 admin admin] PATH_INFO: `/v3/projects/ba02dca1ca1f40ec8e4b909eb3967d7f` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:22.465679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eec2ac-846f-4e73-9218-049dabcbbdb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba02dca1ca1f40ec8e4b909eb3967d7f)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:22.468412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eec2ac-846f-4e73-9218-049dabcbbdb8 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:22.643987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c186f-7401-492d-80cc-f006f3cdb74b service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:22.649975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-554c186f-7401-492d-80cc-f006f3cdb74b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:22.660694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f2eec2ac-846f-4e73-9218-049dabcbbdb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba02dca1ca1f40ec8e4b909eb3967d7f', 'request_id': 'req-f2eec2ac-846f-4e73-9218-049dabcbbdb8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:22.694360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f2eec2ac-846f-4e73-9218-049dabcbbdb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba02dca1ca1f40ec8e4b909eb3967d7f', 'request_id': 'req-f2eec2ac-846f-4e73-9218-049dabcbbdb8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:22.721530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f2eec2ac-846f-4e73-9218-049dabcbbdb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba02dca1ca1f40ec8e4b909eb3967d7f', 'request_id': 'req-f2eec2ac-846f-4e73-9218-049dabcbbdb8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:22.739774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c186f-7401-492d-80cc-f006f3cdb74b service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:22.751455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f2eec2ac-846f-4e73-9218-049dabcbbdb8 admin admin] The token cache is being invalidate because project ba02dca1ca1f40ec8e4b909eb3967d7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:22.763839 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f2eec2ac-846f-4e73-9218-049dabcbbdb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:22.765100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f2eec2ac-846f-4e73-9218-049dabcbbdb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:22.773363 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1084/2150] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:22 2022] DELETE /identity/v3/projects/ba02dca1ca1f40ec8e4b909eb3967d7f => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:22.786534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15f005-7cb7-44f5-9f7c-7567d178bc30 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:22.787696 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15f005-7cb7-44f5-9f7c-7567d178bc30 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:22.788244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15f005-7cb7-44f5-9f7c-7567d178bc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:22.843334 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ab15f005-7cb7-44f5-9f7c-7567d178bc30 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:22.844831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ab15f005-7cb7-44f5-9f7c-7567d178bc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:22.953503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c20e6d-3023-45e2-88d1-e02407789ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:22.955202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20e6d-3023-45e2-88d1-e02407789ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:22.955709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20e6d-3023-45e2-88d1-e02407789ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:22.956200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20e6d-3023-45e2-88d1-e02407789ebb admin admin] PATH_INFO: `/v3/projects/1de2da2eb4944bf78b3cc97ea215efd4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:22.958381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c20e6d-3023-45e2-88d1-e02407789ebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1de2da2eb4944bf78b3cc97ea215efd4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:22.962719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c20e6d-3023-45e2-88d1-e02407789ebb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:22.985425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1085/2151] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:22.985425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15f005-7cb7-44f5-9f7c-7567d178bc30 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:22.987152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ab15f005-7cb7-44f5-9f7c-7567d178bc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:23.002403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15f005-7cb7-44f5-9f7c-7567d178bc30 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.008745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.010309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.010595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.011115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] PATH_INFO: `/v3/users/6c234ceb8057478a88ef4ca0367afca6` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.012592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c234ceb8057478a88ef4ca0367afca6)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.014912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.066870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-41c20e6d-3023-45e2-88d1-e02407789ebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1de2da2eb4944bf78b3cc97ea215efd4', 'request_id': 'req-41c20e6d-3023-45e2-88d1-e02407789ebb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.078153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-41c20e6d-3023-45e2-88d1-e02407789ebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1de2da2eb4944bf78b3cc97ea215efd4', 'request_id': 'req-41c20e6d-3023-45e2-88d1-e02407789ebb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.090703 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-41c20e6d-3023-45e2-88d1-e02407789ebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1de2da2eb4944bf78b3cc97ea215efd4', 'request_id': 'req-41c20e6d-3023-45e2-88d1-e02407789ebb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.093995 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c234ceb8057478a88ef4ca0367afca6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.105299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c234ceb8057478a88ef4ca0367afca6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.106742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c234ceb8057478a88ef4ca0367afca6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.107104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-41c20e6d-3023-45e2-88d1-e02407789ebb admin admin] The token cache is being invalidate because project 1de2da2eb4944bf78b3cc97ea215efd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:23.107417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-41c20e6d-3023-45e2-88d1-e02407789ebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.108100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-41c20e6d-3023-45e2-88d1-e02407789ebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.109863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:23.113856 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1067/2152] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:22 2022] DELETE /identity/v3/projects/1de2da2eb4944bf78b3cc97ea215efd4 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:23.115944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbc784-cb40-440e-90b1-ea8219cf01a7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.116630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbc784-cb40-440e-90b1-ea8219cf01a7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.117064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbc784-cb40-440e-90b1-ea8219cf01a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.122076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c234ceb8057478a88ef4ca0367afca6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.123677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c234ceb8057478a88ef4ca0367afca6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.142031 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-19dbc784-cb40-440e-90b1-ea8219cf01a7 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:23.144114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-19dbc784-cb40-440e-90b1-ea8219cf01a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:23.164386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1086/2153] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:22 2022] DELETE /identity/v3/users/6c234ceb8057478a88ef4ca0367afca6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:23.166071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.169347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:23.213764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:23.256083 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1068/2154] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:23.260406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef8101-01ac-46a8-831d-ad00227a12a0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.260406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef8101-01ac-46a8-831d-ad00227a12a0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.260406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef8101-01ac-46a8-831d-ad00227a12a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.278187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-90ef8101-01ac-46a8-831d-ad00227a12a0 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:23.279495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-90ef8101-01ac-46a8-831d-ad00227a12a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:23.292074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1069/2155] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:23.293039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef8101-01ac-46a8-831d-ad00227a12a0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.294968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ab925-0c2f-4b3a-b079-902cab158da9 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.304683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.305585 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef8101-01ac-46a8-831d-ad00227a12a0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.306898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.307541 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.308266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] PATH_INFO: `/v3/users/f4b49d0ecea44e61bb9eccf5490a0ac2` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.311151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b49d0ecea44e61bb9eccf5490a0ac2)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.312268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaed828-3de2-45dd-86e1-f68d7958c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.313002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaed828-3de2-45dd-86e1-f68d7958c23a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.313394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaed828-3de2-45dd-86e1-f68d7958c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.313751 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaed828-3de2-45dd-86e1-f68d7958c23a admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.314589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaed828-3de2-45dd-86e1-f68d7958c23a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:23.314900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaed828-3de2-45dd-86e1-f68d7958c23a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.315742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.316803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaed828-3de2-45dd-86e1-f68d7958c23a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.332369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1070/2156] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:23.332369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaed828-3de2-45dd-86e1-f68d7958c23a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.341467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1aaed828-3de2-45dd-86e1-f68d7958c23a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:23.411754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b49d0ecea44e61bb9eccf5490a0ac2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.427938 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b49d0ecea44e61bb9eccf5490a0ac2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.429339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b49d0ecea44e61bb9eccf5490a0ac2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.440025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b49d0ecea44e61bb9eccf5490a0ac2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.440920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b49d0ecea44e61bb9eccf5490a0ac2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:23.442726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaed828-3de2-45dd-86e1-f68d7958c23a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.449057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83f67e-90f5-423f-84a3-a473ca7970cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.450841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83f67e-90f5-423f-84a3-a473ca7970cb service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.451605 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83f67e-90f5-423f-84a3-a473ca7970cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.452017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83f67e-90f5-423f-84a3-a473ca7970cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.454693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ab83f67e-90f5-423f-84a3-a473ca7970cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:23.471220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1087/2157] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:23 2022] DELETE /identity/v3/users/f4b49d0ecea44e61bb9eccf5490a0ac2 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:23.472634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.474262 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:23.478299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83f67e-90f5-423f-84a3-a473ca7970cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.480761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83f67e-90f5-423f-84a3-a473ca7970cb service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.498917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1071/2158] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:23.581364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:23.603888 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32790f5b-8549-40c1-9564-35d741af3eb8 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.610939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1c327-ade5-4fbc-a55b-16fc0fd752bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.611742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1c327-ade5-4fbc-a55b-16fc0fd752bf admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.612039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1c327-ade5-4fbc-a55b-16fc0fd752bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.612320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1c327-ade5-4fbc-a55b-16fc0fd752bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.613005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1c327-ade5-4fbc-a55b-16fc0fd752bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.615507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1c327-ade5-4fbc-a55b-16fc0fd752bf admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.659997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1088/2159] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:23 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:23.668140 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83f67e-90f5-423f-84a3-a473ca7970cb service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.682013 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83f67e-90f5-423f-84a3-a473ca7970cb service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.841525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d16211-e547-4e53-b014-63a6f7d81fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.842310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d16211-e547-4e53-b014-63a6f7d81fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.843072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d16211-e547-4e53-b014-63a6f7d81fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.843072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d16211-e547-4e53-b014-63a6f7d81fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.843714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d16211-e547-4e53-b014-63a6f7d81fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.848558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d16211-e547-4e53-b014-63a6f7d81fb3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.917026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1072/2160] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:23 2022] POST /identity/v3/users => generated 287 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:23.919998 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1c327-ade5-4fbc-a55b-16fc0fd752bf admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.954592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1c327-ade5-4fbc-a55b-16fc0fd752bf admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.968839 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70c6d0-222c-4603-8f39-50445fb9051a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.970503 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70c6d0-222c-4603-8f39-50445fb9051a admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.971029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70c6d0-222c-4603-8f39-50445fb9051a admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.971517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70c6d0-222c-4603-8f39-50445fb9051a admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.973013 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70c6d0-222c-4603-8f39-50445fb9051a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.977111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70c6d0-222c-4603-8f39-50445fb9051a admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:24.019051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1089/2161] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:24.021112 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d16211-e547-4e53-b014-63a6f7d81fb3 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:24.043991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d16211-e547-4e53-b014-63a6f7d81fb3 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:24.054266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673a84f-1d21-48ad-9765-771767f95f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:24.055364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673a84f-1d21-48ad-9765-771767f95f76 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.056434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673a84f-1d21-48ad-9765-771767f95f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.056972 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673a84f-1d21-48ad-9765-771767f95f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.065391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673a84f-1d21-48ad-9765-771767f95f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:24.069637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673a84f-1d21-48ad-9765-771767f95f76 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:24.094777 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1073/2162] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:24.101754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70c6d0-222c-4603-8f39-50445fb9051a admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:24.127741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70c6d0-222c-4603-8f39-50445fb9051a admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:24.140325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b14054-8442-4213-8688-0c0c562790a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:24.142057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b14054-8442-4213-8688-0c0c562790a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.142800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b14054-8442-4213-8688-0c0c562790a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.143274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b14054-8442-4213-8688-0c0c562790a2 admin admin] PATH_INFO: `/v3/projects/58636dfe0f8c463bab0031b5d6734bd9/users/35f252f0338547aea73d1eb7c81ca199/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.177353 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b14054-8442-4213-8688-0c0c562790a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58636dfe0f8c463bab0031b5d6734bd9, user_id=35f252f0338547aea73d1eb7c81ca199, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:24.184177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b14054-8442-4213-8688-0c0c562790a2 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:24.217385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1090/2163] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:24 2022] PUT /identity/v3/projects/58636dfe0f8c463bab0031b5d6734bd9/users/35f252f0338547aea73d1eb7c81ca199/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:24.229006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107493a-da93-47e4-9349-100209b28179 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.229633 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107493a-da93-47e4-9349-100209b28179 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.230375 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107493a-da93-47e4-9349-100209b28179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.281642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1107493a-da93-47e4-9349-100209b28179 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:24.284866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1107493a-da93-47e4-9349-100209b28179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:24.331497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b14054-8442-4213-8688-0c0c562790a2 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:24.339682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-65b14054-8442-4213-8688-0c0c562790a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:24.452313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:24.526151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1074/2164] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:24.533696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107493a-da93-47e4-9349-100209b28179 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:24.538527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1107493a-da93-47e4-9349-100209b28179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:24.576019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107493a-da93-47e4-9349-100209b28179 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:24.588805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:24.591655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.592328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.593001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] PATH_INFO: `/v3/projects/62849039c2504feab54537898697ee61` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.596535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] RBAC: Authorizing `identity:delete_project(project_id=62849039c2504feab54537898697ee61)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:24.601472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:24.614687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b14054-8442-4213-8688-0c0c562790a2 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:24.635099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:24.636414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.636968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.637484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] PATH_INFO: `/v3/projects/4377e03834384210b72dfd606c57b690` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.641842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4377e03834384210b72dfd606c57b690)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:24.645616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:24.732123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62849039c2504feab54537898697ee61', 'request_id': 'req-18f461f0-e5ca-4851-9920-7ab833b2c12e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.744271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62849039c2504feab54537898697ee61', 'request_id': 'req-18f461f0-e5ca-4851-9920-7ab833b2c12e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.764582 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62849039c2504feab54537898697ee61', 'request_id': 'req-18f461f0-e5ca-4851-9920-7ab833b2c12e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.786387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] The token cache is being invalidate because project 62849039c2504feab54537898697ee61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:24.786796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.787089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4377e03834384210b72dfd606c57b690', 'request_id': 'req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.788123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.792529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1075/2165] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:24 2022] DELETE /identity/v3/projects/62849039c2504feab54537898697ee61 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:24.793913 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:24.796288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:24.804727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4377e03834384210b72dfd606c57b690', 'request_id': 'req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.818071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4377e03834384210b72dfd606c57b690', 'request_id': 'req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.862080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] The token cache is being invalidate because project 4377e03834384210b72dfd606c57b690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:24.863311 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.865020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:24.868707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1091/2166] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:24 2022] DELETE /identity/v3/projects/4377e03834384210b72dfd606c57b690 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:24.874380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:24.879555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:24.973208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f461f0-e5ca-4851-9920-7ab833b2c12e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:24.980200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:25.082996 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be63b0e-1fc7-487b-ac0d-ce0517c947cf admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:25.100867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:25.101981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:25.102274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:25.103153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] PATH_INFO: `/v3/users/a7008ea98f3e487caeec8efd96082160` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:25.106757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7008ea98f3e487caeec8efd96082160)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:25.116364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:25.117831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730269b5-e3ce-4cd8-8387-33d433a703df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:25.121457 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730269b5-e3ce-4cd8-8387-33d433a703df service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:25.121939 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730269b5-e3ce-4cd8-8387-33d433a703df service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:25.123287 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730269b5-e3ce-4cd8-8387-33d433a703df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:25.126959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-730269b5-e3ce-4cd8-8387-33d433a703df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:25.158181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730269b5-e3ce-4cd8-8387-33d433a703df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:25.162421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730269b5-e3ce-4cd8-8387-33d433a703df service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:25.195361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1076/2167] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:25.202874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730269b5-e3ce-4cd8-8387-33d433a703df service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:25.208569 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-730269b5-e3ce-4cd8-8387-33d433a703df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:25.238783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730269b5-e3ce-4cd8-8387-33d433a703df service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:25.248385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:25.249275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:25.249560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:25.249836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] PATH_INFO: `/v3/users/b441530e4b24471db57d48d6a4a6fc94` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:25.251188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b441530e4b24471db57d48d6a4a6fc94)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:25.253356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:25.277550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:25.292928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7008ea98f3e487caeec8efd96082160'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:25.335845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7008ea98f3e487caeec8efd96082160'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:25.337217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7008ea98f3e487caeec8efd96082160'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:25.365628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7008ea98f3e487caeec8efd96082160'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:25.368301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7008ea98f3e487caeec8efd96082160'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:25.390989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b441530e4b24471db57d48d6a4a6fc94'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:25.419586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b441530e4b24471db57d48d6a4a6fc94'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:25.439383 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1092/2168] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:24 2022] DELETE /identity/v3/users/a7008ea98f3e487caeec8efd96082160 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:25.467954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b441530e4b24471db57d48d6a4a6fc94'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:25.494727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b441530e4b24471db57d48d6a4a6fc94'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:25.497606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b441530e4b24471db57d48d6a4a6fc94'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:25.512470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1077/2169] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:25 2022] DELETE /identity/v3/users/b441530e4b24471db57d48d6a4a6fc94 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:26.780293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:26.786080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:26.819817 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:26.821859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:26.973129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1fc73-66c0-4446-b231-af73fdc55891 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:26.990563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:26.991680 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e38c0-1350-430e-b3f0-37f31fc951a4 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:26.991971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.992399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.992850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] PATH_INFO: `/v3/projects/a5c24d3205d2434a9b09e9368d9fa965` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.995833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5c24d3205d2434a9b09e9368d9fa965)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:26.998726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.002703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.004689 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.005359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.005876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14 admin admin] PATH_INFO: `/v3/projects/43dc58493f0443b1a7d657f3d005b929` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.008919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43dc58493f0443b1a7d657f3d005b929)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.013841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.090003 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5c24d3205d2434a9b09e9368d9fa965', 'request_id': 'req-c68d9f20-065e-4f01-b6e8-292dfdd15815'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.103999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5c24d3205d2434a9b09e9368d9fa965', 'request_id': 'req-c68d9f20-065e-4f01-b6e8-292dfdd15815'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.117513 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5c24d3205d2434a9b09e9368d9fa965', 'request_id': 'req-c68d9f20-065e-4f01-b6e8-292dfdd15815'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.146549 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] The token cache is being invalidate because project a5c24d3205d2434a9b09e9368d9fa965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:27.147061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.148558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.152405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1078/2170] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:26 2022] DELETE /identity/v3/projects/a5c24d3205d2434a9b09e9368d9fa965 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:27.153703 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.155310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:27.161092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43dc58493f0443b1a7d657f3d005b929', 'request_id': 'req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.171975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:27.179046 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43dc58493f0443b1a7d657f3d005b929', 'request_id': 'req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.198469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43dc58493f0443b1a7d657f3d005b929', 'request_id': 'req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.238326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:27.247916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14 admin admin] The token cache is being invalidate because project 43dc58493f0443b1a7d657f3d005b929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:27.248728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.250694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-5b887f8b-3fc0-4991-94cf-fc9d29a11e14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.256816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1093/2171] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:26 2022] DELETE /identity/v3/projects/43dc58493f0443b1a7d657f3d005b929 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:27.269969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9e3bb-1d24-4cc7-a435-42aff65896a7 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.271064 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9e3bb-1d24-4cc7-a435-42aff65896a7 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.271845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9e3bb-1d24-4cc7-a435-42aff65896a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.320138 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-82a9e3bb-1d24-4cc7-a435-42aff65896a7 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:27.324304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-82a9e3bb-1d24-4cc7-a435-42aff65896a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:27.454389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d9f20-065e-4f01-b6e8-292dfdd15815 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.553823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1094/2172] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:27.556074 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567de66-b03d-4ea1-bf32-b54b17cc34ed None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.556597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567de66-b03d-4ea1-bf32-b54b17cc34ed None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.558045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567de66-b03d-4ea1-bf32-b54b17cc34ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.611414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-3567de66-b03d-4ea1-bf32-b54b17cc34ed None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:27.612884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3567de66-b03d-4ea1-bf32-b54b17cc34ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:27.663066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1095/2173] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:27.664704 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cebe33-e5c6-47fe-99ff-424a4922419e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.666083 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cebe33-e5c6-47fe-99ff-424a4922419e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.666444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cebe33-e5c6-47fe-99ff-424a4922419e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.666745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567de66-b03d-4ea1-bf32-b54b17cc34ed None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.666859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cebe33-e5c6-47fe-99ff-424a4922419e admin admin] PATH_INFO: `/v3/users/f2a73df58810444697a41f8e851d8f0d` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.668199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cebe33-e5c6-47fe-99ff-424a4922419e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2a73df58810444697a41f8e851d8f0d)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.670337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cebe33-e5c6-47fe-99ff-424a4922419e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.704342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567de66-b03d-4ea1-bf32-b54b17cc34ed None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.720961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e6f69-2515-4122-a33e-9cc0fea5a693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.721802 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e6f69-2515-4122-a33e-9cc0fea5a693 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.722499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e6f69-2515-4122-a33e-9cc0fea5a693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.723615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e6f69-2515-4122-a33e-9cc0fea5a693 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.725435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e6f69-2515-4122-a33e-9cc0fea5a693 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:27.727192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e6f69-2515-4122-a33e-9cc0fea5a693 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.731972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e6f69-2515-4122-a33e-9cc0fea5a693 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.760052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1096/2174] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:27.766489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e6f69-2515-4122-a33e-9cc0fea5a693 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.785433 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e6f69-2515-4122-a33e-9cc0fea5a693 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.791964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ae1d3-69f9-4aa7-9117-31f7a06336df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.792871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ae1d3-69f9-4aa7-9117-31f7a06336df admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.793264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ae1d3-69f9-4aa7-9117-31f7a06336df admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.793746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ae1d3-69f9-4aa7-9117-31f7a06336df admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.794603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ae1d3-69f9-4aa7-9117-31f7a06336df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.796784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ae1d3-69f9-4aa7-9117-31f7a06336df admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.803760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c5cebe33-e5c6-47fe-99ff-424a4922419e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2a73df58810444697a41f8e851d8f0d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.824512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c5cebe33-e5c6-47fe-99ff-424a4922419e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2a73df58810444697a41f8e851d8f0d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.833866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1097/2175] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:27 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:27.843280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ae1d3-69f9-4aa7-9117-31f7a06336df admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.859907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c5cebe33-e5c6-47fe-99ff-424a4922419e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2a73df58810444697a41f8e851d8f0d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.886265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c5cebe33-e5c6-47fe-99ff-424a4922419e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2a73df58810444697a41f8e851d8f0d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.888077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c5cebe33-e5c6-47fe-99ff-424a4922419e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2a73df58810444697a41f8e851d8f0d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:27.906544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1079/2176] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:27 2022] DELETE /identity/v3/users/f2a73df58810444697a41f8e851d8f0d => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:27.911338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c749b-c35e-41a8-a49d-f06a151dcf4b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.913244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c749b-c35e-41a8-a49d-f06a151dcf4b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.914071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c749b-c35e-41a8-a49d-f06a151dcf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.950566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d18c749b-c35e-41a8-a49d-f06a151dcf4b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:27.953078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d18c749b-c35e-41a8-a49d-f06a151dcf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:27.982254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ae1d3-69f9-4aa7-9117-31f7a06336df admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.079935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:28.107586 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:28.130582 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1080/2177] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:28.145402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b4e94-755a-446a-a407-55aa54008d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.146373 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b4e94-755a-446a-a407-55aa54008d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.146750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b4e94-755a-446a-a407-55aa54008d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.147109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b4e94-755a-446a-a407-55aa54008d59 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.148420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c749b-c35e-41a8-a49d-f06a151dcf4b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.148837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b4e94-755a-446a-a407-55aa54008d59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.153010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d18c749b-c35e-41a8-a49d-f06a151dcf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:28.156237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b4e94-755a-446a-a407-55aa54008d59 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.207128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1098/2178] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:27 2022] POST /identity/v3/users => generated 293 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:28.212318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b4e94-755a-446a-a407-55aa54008d59 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.215760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f57b4e94-755a-446a-a407-55aa54008d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:28.244437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b4e94-755a-446a-a407-55aa54008d59 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.278786 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bfb21-2481-4110-93f5-31dd14791de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.280197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bfb21-2481-4110-93f5-31dd14791de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.280719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bfb21-2481-4110-93f5-31dd14791de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.281082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bfb21-2481-4110-93f5-31dd14791de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.282849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c749b-c35e-41a8-a49d-f06a151dcf4b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.283430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bfb21-2481-4110-93f5-31dd14791de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.289162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bfb21-2481-4110-93f5-31dd14791de4 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.291140 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de835fde-a3b4-4f29-aa76-95c61fcd00ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.292556 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de835fde-a3b4-4f29-aa76-95c61fcd00ee service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.293134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de835fde-a3b4-4f29-aa76-95c61fcd00ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.293629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de835fde-a3b4-4f29-aa76-95c61fcd00ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.318513 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1099/2179] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:28.323293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18517b8f-d460-4a1e-b954-165e18255666 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.323695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18517b8f-d460-4a1e-b954-165e18255666 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.323884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18517b8f-d460-4a1e-b954-165e18255666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.328414 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de835fde-a3b4-4f29-aa76-95c61fcd00ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.330598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de835fde-a3b4-4f29-aa76-95c61fcd00ee service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.361686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1081/2180] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:28.362676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de835fde-a3b4-4f29-aa76-95c61fcd00ee service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.382781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de835fde-a3b4-4f29-aa76-95c61fcd00ee service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.392572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-18517b8f-d460-4a1e-b954-165e18255666 None None] MFA Rules not processed for user `35f252f0338547aea73d1eb7c81ca199`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:28.395719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988c229-1a4a-42c3-8dea-997947842f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.396292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-18517b8f-d460-4a1e-b954-165e18255666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:28.397215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988c229-1a4a-42c3-8dea-997947842f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.397842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988c229-1a4a-42c3-8dea-997947842f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.398577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988c229-1a4a-42c3-8dea-997947842f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.401291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988c229-1a4a-42c3-8dea-997947842f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.407119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988c229-1a4a-42c3-8dea-997947842f58 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.476110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1082/2181] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:28.477162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988c229-1a4a-42c3-8dea-997947842f58 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.485930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5988c229-1a4a-42c3-8dea-997947842f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:28.542693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988c229-1a4a-42c3-8dea-997947842f58 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.565548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c9da1-7efe-460d-b31d-00cda4fe206f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.566750 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c9da1-7efe-460d-b31d-00cda4fe206f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.567178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c9da1-7efe-460d-b31d-00cda4fe206f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.567589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c9da1-7efe-460d-b31d-00cda4fe206f admin admin] PATH_INFO: `/v3/users/a0e13c7e9e3442dd923e790c6c6259ba` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.570860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c9da1-7efe-460d-b31d-00cda4fe206f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e13c7e9e3442dd923e790c6c6259ba)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.573665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c9da1-7efe-460d-b31d-00cda4fe206f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.626688 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1100/2182] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:28.644817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18517b8f-d460-4a1e-b954-165e18255666 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.686890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18517b8f-d460-4a1e-b954-165e18255666 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.703099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c4e1c-bcb4-404c-8195-c1c0d1bc611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.704627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c4e1c-bcb4-404c-8195-c1c0d1bc611b admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.705329 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c4e1c-bcb4-404c-8195-c1c0d1bc611b admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.721497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c4e1c-bcb4-404c-8195-c1c0d1bc611b admin admin] PATH_INFO: `/v3/projects/1aac5c827bca4571bc0191a20846041f/users/2ba346eef71c47a7819896c8cc6a02fe/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.783489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c4e1c-bcb4-404c-8195-c1c0d1bc611b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aac5c827bca4571bc0191a20846041f, user_id=2ba346eef71c47a7819896c8cc6a02fe, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.800597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c4e1c-bcb4-404c-8195-c1c0d1bc611b admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.812976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-027c9da1-7efe-460d-b31d-00cda4fe206f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e13c7e9e3442dd923e790c6c6259ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:28.839644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1101/2183] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:28 2022] PUT /identity/v3/projects/1aac5c827bca4571bc0191a20846041f/users/2ba346eef71c47a7819896c8cc6a02fe/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:28.840901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c4e1c-bcb4-404c-8195-c1c0d1bc611b admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.845646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-027c9da1-7efe-460d-b31d-00cda4fe206f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e13c7e9e3442dd923e790c6c6259ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:28.850324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-578c4e1c-bcb4-404c-8195-c1c0d1bc611b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:28.890785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-027c9da1-7efe-460d-b31d-00cda4fe206f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e13c7e9e3442dd923e790c6c6259ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:28.918452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-027c9da1-7efe-460d-b31d-00cda4fe206f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e13c7e9e3442dd923e790c6c6259ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:28.919516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-027c9da1-7efe-460d-b31d-00cda4fe206f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e13c7e9e3442dd923e790c6c6259ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:28.935603 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1083/2184] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:28 2022] DELETE /identity/v3/users/a0e13c7e9e3442dd923e790c6c6259ba => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:28.938489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c673e28-82c8-4f73-ad1b-eb18095517e0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.938996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c673e28-82c8-4f73-ad1b-eb18095517e0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.939477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c673e28-82c8-4f73-ad1b-eb18095517e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.009309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9c673e28-82c8-4f73-ad1b-eb18095517e0 None None] MFA Rules not processed for user `2ba346eef71c47a7819896c8cc6a02fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:29.012005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9c673e28-82c8-4f73-ad1b-eb18095517e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:29.061731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c4e1c-bcb4-404c-8195-c1c0d1bc611b admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.143417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:29.172115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:29.178918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7acf37-9ea2-4bf6-a936-e531580dd4e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.181861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7acf37-9ea2-4bf6-a936-e531580dd4e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.182357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7acf37-9ea2-4bf6-a936-e531580dd4e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.183307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7acf37-9ea2-4bf6-a936-e531580dd4e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.190118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fe7acf37-9ea2-4bf6-a936-e531580dd4e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:29.211972 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1084/2185] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:29.215333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c1913-9754-47cd-9e15-01c76470cece None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.216249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c1913-9754-47cd-9e15-01c76470cece None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.217062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c1913-9754-47cd-9e15-01c76470cece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.253546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5c7c1913-9754-47cd-9e15-01c76470cece None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:29.256426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5c7c1913-9754-47cd-9e15-01c76470cece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:29.301338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7acf37-9ea2-4bf6-a936-e531580dd4e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.304430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7acf37-9ea2-4bf6-a936-e531580dd4e6 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.318927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1102/2186] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:29.319989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7acf37-9ea2-4bf6-a936-e531580dd4e6 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.323186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fe7acf37-9ea2-4bf6-a936-e531580dd4e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:29.433383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1085/2187] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:29.434555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c1913-9754-47cd-9e15-01c76470cece None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.444645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5c7c1913-9754-47cd-9e15-01c76470cece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:29.475674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7acf37-9ea2-4bf6-a936-e531580dd4e6 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.479007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c1913-9754-47cd-9e15-01c76470cece None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.487063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.488746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.489298 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.489842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] PATH_INFO: `/v3/projects/2ffb60a72ee543deb455efffff800417` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.493348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffb60a72ee543deb455efffff800417)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.494056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77033909-cada-42ec-afab-e56aa90c2b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.494927 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77033909-cada-42ec-afab-e56aa90c2b2d admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.497734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77033909-cada-42ec-afab-e56aa90c2b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.498175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77033909-cada-42ec-afab-e56aa90c2b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.499723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77033909-cada-42ec-afab-e56aa90c2b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.499849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.505357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77033909-cada-42ec-afab-e56aa90c2b2d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.587360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1086/2188] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:29 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:29.589759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77033909-cada-42ec-afab-e56aa90c2b2d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.593412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-77033909-cada-42ec-afab-e56aa90c2b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:29.666688 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffb60a72ee543deb455efffff800417', 'request_id': 'req-15291635-e65d-47f1-9def-f318b7eb3aaf'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:29.688722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77033909-cada-42ec-afab-e56aa90c2b2d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.693164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffb60a72ee543deb455efffff800417', 'request_id': 'req-15291635-e65d-47f1-9def-f318b7eb3aaf'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:29.704117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffb60a72ee543deb455efffff800417', 'request_id': 'req-15291635-e65d-47f1-9def-f318b7eb3aaf'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:29.773507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] The token cache is being invalidate because project 2ffb60a72ee543deb455efffff800417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:29.774242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:29.775998 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:29.778980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1103/2189] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:29 2022] DELETE /identity/v3/projects/2ffb60a72ee543deb455efffff800417 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:29.781103 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.783164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:29.851709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:29.874156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a32b3e-a0f6-4e9b-a28c-a64d0416d278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.875853 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a32b3e-a0f6-4e9b-a28c-a64d0416d278 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.878645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a32b3e-a0f6-4e9b-a28c-a64d0416d278 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.879216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a32b3e-a0f6-4e9b-a28c-a64d0416d278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.892648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-02a32b3e-a0f6-4e9b-a28c-a64d0416d278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:29.907048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15291635-e65d-47f1-9def-f318b7eb3aaf admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.916612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602627d4-a83b-4d43-a36b-953dd60cfe3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.917731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602627d4-a83b-4d43-a36b-953dd60cfe3c admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.918298 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602627d4-a83b-4d43-a36b-953dd60cfe3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.918857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602627d4-a83b-4d43-a36b-953dd60cfe3c admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.920018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602627d4-a83b-4d43-a36b-953dd60cfe3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.922569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602627d4-a83b-4d43-a36b-953dd60cfe3c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.950080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a32b3e-a0f6-4e9b-a28c-a64d0416d278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.957624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1104/2190] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:29 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:29.958065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a32b3e-a0f6-4e9b-a28c-a64d0416d278 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.959688 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872181-c473-43aa-bb02-a063ca6add6b None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.960149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872181-c473-43aa-bb02-a063ca6add6b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.960598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872181-c473-43aa-bb02-a063ca6add6b None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.962928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1105/2191] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 22:35:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:29.963964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36872181-c473-43aa-bb02-a063ca6add6b None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.966630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-36872181-c473-43aa-bb02-a063ca6add6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:30.003307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1087/2192] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:30.009711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a32b3e-a0f6-4e9b-a28c-a64d0416d278 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.015773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36872181-c473-43aa-bb02-a063ca6add6b None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.043529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.045849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a32b3e-a0f6-4e9b-a28c-a64d0416d278 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.048248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.048547 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.048974 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] PATH_INFO: `/v3/users/c43bf6734c984c6e9d189660f903958b` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.050470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c43bf6734c984c6e9d189660f903958b)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.052644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.066318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992c8c5-1d83-438d-83f4-75efd3b1440e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.069508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992c8c5-1d83-438d-83f4-75efd3b1440e admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.071937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992c8c5-1d83-438d-83f4-75efd3b1440e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.075916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992c8c5-1d83-438d-83f4-75efd3b1440e admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.077501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992c8c5-1d83-438d-83f4-75efd3b1440e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.081302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992c8c5-1d83-438d-83f4-75efd3b1440e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.098260 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1088/2193] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:30.100175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d2f37-011e-4dad-a62d-5ae2b2095a03 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.100664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d2f37-011e-4dad-a62d-5ae2b2095a03 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.101920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d2f37-011e-4dad-a62d-5ae2b2095a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.130942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-da7d2f37-011e-4dad-a62d-5ae2b2095a03 None None] MFA Rules not processed for user `705ff0e73e95499e873b1df2a24fefe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:30.133273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-da7d2f37-011e-4dad-a62d-5ae2b2095a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:30.137500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c43bf6734c984c6e9d189660f903958b'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.140004 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:30.165558 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c43bf6734c984c6e9d189660f903958b'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.166292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c43bf6734c984c6e9d189660f903958b'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.179061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c43bf6734c984c6e9d189660f903958b'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.180050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c43bf6734c984c6e9d189660f903958b'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.216471 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1106/2194] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:29 2022] DELETE /identity/v3/users/c43bf6734c984c6e9d189660f903958b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:30.217710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.219324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:30.271002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1089/2195] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 22:35:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:30.283341 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d2f37-011e-4dad-a62d-5ae2b2095a03 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.285313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-da7d2f37-011e-4dad-a62d-5ae2b2095a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:30.313720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d2f37-011e-4dad-a62d-5ae2b2095a03 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.400285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dfa04-b124-4157-9877-d01ee2594ef8 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.417318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ae7ab-0309-48d8-a9cf-8436ba4ecb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.419239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ae7ab-0309-48d8-a9cf-8436ba4ecb09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.423966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ae7ab-0309-48d8-a9cf-8436ba4ecb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.424514 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ae7ab-0309-48d8-a9cf-8436ba4ecb09 admin admin] PATH_INFO: `/v3/projects/14b4a2eba9bc497ebea653449476e500/users/7851b6ef1a2c480899ed46dac4f4d634/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.454897 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ae7ab-0309-48d8-a9cf-8436ba4ecb09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b4a2eba9bc497ebea653449476e500, user_id=7851b6ef1a2c480899ed46dac4f4d634, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.460071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ae7ab-0309-48d8-a9cf-8436ba4ecb09 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.490021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1107/2196] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:30 2022] PUT /identity/v3/projects/14b4a2eba9bc497ebea653449476e500/users/7851b6ef1a2c480899ed46dac4f4d634/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:30.495490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ae7ab-0309-48d8-a9cf-8436ba4ecb09 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.533250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d6ce1-2f88-49f2-ab71-335200a16550 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.535189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d6ce1-2f88-49f2-ab71-335200a16550 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.535907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d6ce1-2f88-49f2-ab71-335200a16550 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.536655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d6ce1-2f88-49f2-ab71-335200a16550 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.546088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:30.557630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d6ce1-2f88-49f2-ab71-335200a16550 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.559419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d6ce1-2f88-49f2-ab71-335200a16550 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.581595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1090/2197] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:30.582895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d6ce1-2f88-49f2-ab71-335200a16550 service placement] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.584794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a56d6ce1-2f88-49f2-ab71-335200a16550 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:30.626010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d6ce1-2f88-49f2-ab71-335200a16550 service placement] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.672939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ae7ab-0309-48d8-a9cf-8436ba4ecb09 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.678964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09e45b-f5b9-4e9b-b76c-a2776202c5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.679733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e45b-f5b9-4e9b-b76c-a2776202c5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.680048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e45b-f5b9-4e9b-b76c-a2776202c5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.681270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e45b-f5b9-4e9b-b76c-a2776202c5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.682007 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09e45b-f5b9-4e9b-b76c-a2776202c5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.684502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09e45b-f5b9-4e9b-b76c-a2776202c5e6 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.698118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1108/2198] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:30.698118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09e45b-f5b9-4e9b-b76c-a2776202c5e6 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.703499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bb09e45b-f5b9-4e9b-b76c-a2776202c5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:30.725249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.726189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.726475 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.726755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] PATH_INFO: `/v3/projects/aa3208211a214b4a9c7face1a1fed6f0` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.728698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09e45b-f5b9-4e9b-b76c-a2776202c5e6 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.728882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa3208211a214b4a9c7face1a1fed6f0)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.730808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.734905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.735811 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.736102 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.736418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487 admin admin] PATH_INFO: `/v3/projects/156f20101bf144d4accc3610faca358e` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.737872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=156f20101bf144d4accc3610faca358e)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.739850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.788840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:30.806849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3208211a214b4a9c7face1a1fed6f0', 'request_id': 'req-6e8ff07d-7b04-4514-89bb-751e64768e65'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.817741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3208211a214b4a9c7face1a1fed6f0', 'request_id': 'req-6e8ff07d-7b04-4514-89bb-751e64768e65'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.825481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '156f20101bf144d4accc3610faca358e', 'request_id': 'req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.828438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3208211a214b4a9c7face1a1fed6f0', 'request_id': 'req-6e8ff07d-7b04-4514-89bb-751e64768e65'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.841642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '156f20101bf144d4accc3610faca358e', 'request_id': 'req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.845148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] The token cache is being invalidate because project aa3208211a214b4a9c7face1a1fed6f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:30.845441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.846148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.847780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1091/2199] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:30 2022] DELETE /identity/v3/projects/aa3208211a214b4a9c7face1a1fed6f0 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:30.848593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.850047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:30.851709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '156f20101bf144d4accc3610faca358e', 'request_id': 'req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.870835 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487 admin admin] The token cache is being invalidate because project 156f20101bf144d4accc3610faca358e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:30.871899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.872310 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-8f5fa6da-87c3-4d9d-9d8f-9353a6dce487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:30.874114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1109/2200] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:30 2022] DELETE /identity/v3/projects/156f20101bf144d4accc3610faca358e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:30.875657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528eb2d-4f20-4a28-9483-1d267ada8b27 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.876112 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528eb2d-4f20-4a28-9483-1d267ada8b27 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.876403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528eb2d-4f20-4a28-9483-1d267ada8b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.900377 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-0528eb2d-4f20-4a28-9483-1d267ada8b27 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:30.901700 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0528eb2d-4f20-4a28-9483-1d267ada8b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:30.958896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ff07d-7b04-4514-89bb-751e64768e65 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.973418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1110/2201] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:30.974398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528eb2d-4f20-4a28-9483-1d267ada8b27 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.975851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0528eb2d-4f20-4a28-9483-1d267ada8b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:30.989842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528eb2d-4f20-4a28-9483-1d267ada8b27 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.996927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029ffe5-7566-4081-ba53-2a50202c9eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.997993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029ffe5-7566-4081-ba53-2a50202c9eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.998369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029ffe5-7566-4081-ba53-2a50202c9eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.998723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029ffe5-7566-4081-ba53-2a50202c9eab admin admin] PATH_INFO: `/v3/users/2f9263206d1749f8ab39b5999689ae35` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.000560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029ffe5-7566-4081-ba53-2a50202c9eab admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9263206d1749f8ab39b5999689ae35)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:31.003927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029ffe5-7566-4081-ba53-2a50202c9eab admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:31.052052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f6c8b-eeae-48d1-b5ff-93632c47c792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.053454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f6c8b-eeae-48d1-b5ff-93632c47c792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.054091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f6c8b-eeae-48d1-b5ff-93632c47c792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.054490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f6c8b-eeae-48d1-b5ff-93632c47c792 admin admin] PATH_INFO: `/v3/projects/14b4a2eba9bc497ebea653449476e500/users/7851b6ef1a2c480899ed46dac4f4d634/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.059261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f6c8b-eeae-48d1-b5ff-93632c47c792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b4a2eba9bc497ebea653449476e500, user_id=7851b6ef1a2c480899ed46dac4f4d634, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:31.064172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f6c8b-eeae-48d1-b5ff-93632c47c792 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:31.065598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:31.087726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1092/2202] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:30 2022] PUT /identity/v3/projects/14b4a2eba9bc497ebea653449476e500/users/7851b6ef1a2c480899ed46dac4f4d634/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:31.089854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbcd54-547a-4cc3-b132-b6c2bc2777cb None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.090314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbcd54-547a-4cc3-b132-b6c2bc2777cb None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.090777 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbcd54-547a-4cc3-b132-b6c2bc2777cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.113970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e029ffe5-7566-4081-ba53-2a50202c9eab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9263206d1749f8ab39b5999689ae35'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:31.123092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b7cbcd54-547a-4cc3-b132-b6c2bc2777cb None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:31.125575 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b7cbcd54-547a-4cc3-b132-b6c2bc2777cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:31.130639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e029ffe5-7566-4081-ba53-2a50202c9eab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9263206d1749f8ab39b5999689ae35'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:31.131746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e029ffe5-7566-4081-ba53-2a50202c9eab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9263206d1749f8ab39b5999689ae35'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:31.147833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e029ffe5-7566-4081-ba53-2a50202c9eab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9263206d1749f8ab39b5999689ae35'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:31.149190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e029ffe5-7566-4081-ba53-2a50202c9eab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9263206d1749f8ab39b5999689ae35'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:31.185018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1111/2203] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:30 2022] DELETE /identity/v3/users/2f9263206d1749f8ab39b5999689ae35 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:31.187224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2557c7a-02d5-4c31-84c8-495fb7303adc None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.187224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2557c7a-02d5-4c31-84c8-495fb7303adc None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.188317 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2557c7a-02d5-4c31-84c8-495fb7303adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.254705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f2557c7a-02d5-4c31-84c8-495fb7303adc None None] MFA Rules not processed for user `7851b6ef1a2c480899ed46dac4f4d634`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:31.256833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f2557c7a-02d5-4c31-84c8-495fb7303adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:31.283011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1093/2204] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:31.284867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbcd54-547a-4cc3-b132-b6c2bc2777cb None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.310807 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbcd54-547a-4cc3-b132-b6c2bc2777cb None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.327641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:31.456058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1112/2205] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:31.457793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e392a-3600-4bf5-a766-e0736ae45abf None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.457956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e392a-3600-4bf5-a766-e0736ae45abf None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.458277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e392a-3600-4bf5-a766-e0736ae45abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.476126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ba318-061d-4693-9434-3fd995bc468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.476852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ba318-061d-4693-9434-3fd995bc468b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.477923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ba318-061d-4693-9434-3fd995bc468b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.478424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ba318-061d-4693-9434-3fd995bc468b admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.479483 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ba318-061d-4693-9434-3fd995bc468b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:31.479796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ba318-061d-4693-9434-3fd995bc468b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:31.481491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ba318-061d-4693-9434-3fd995bc468b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:31.487958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-8e4e392a-3600-4bf5-a766-e0736ae45abf None None] MFA Rules not processed for user `35f252f0338547aea73d1eb7c81ca199`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:31.489582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8e4e392a-3600-4bf5-a766-e0736ae45abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:31.508228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1094/2206] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:31.508228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ba318-061d-4693-9434-3fd995bc468b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.510087 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b31ba318-061d-4693-9434-3fd995bc468b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:31.577902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1113/2207] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:31.577902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e392a-3600-4bf5-a766-e0736ae45abf None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.599154 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e392a-3600-4bf5-a766-e0736ae45abf None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.611361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b50013-2551-405d-97ac-a7b2636707a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.612923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b50013-2551-405d-97ac-a7b2636707a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.613469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b50013-2551-405d-97ac-a7b2636707a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.614529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b50013-2551-405d-97ac-a7b2636707a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.616691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b50013-2551-405d-97ac-a7b2636707a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:31.622021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b50013-2551-405d-97ac-a7b2636707a3 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:31.636183 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ba318-061d-4693-9434-3fd995bc468b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.640834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10a74f-9a1d-499c-b66d-a5d1f272720b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.641228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10a74f-9a1d-499c-b66d-a5d1f272720b service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.641520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10a74f-9a1d-499c-b66d-a5d1f272720b service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.641967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10a74f-9a1d-499c-b66d-a5d1f272720b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.644103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7f10a74f-9a1d-499c-b66d-a5d1f272720b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:31.695859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1114/2208] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:31 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:31.697582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b50013-2551-405d-97ac-a7b2636707a3 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.745866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:31.758155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10a74f-9a1d-499c-b66d-a5d1f272720b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:31.767233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10a74f-9a1d-499c-b66d-a5d1f272720b service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:31.791463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1095/2209] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:31.793402 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10a74f-9a1d-499c-b66d-a5d1f272720b service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.795657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7f10a74f-9a1d-499c-b66d-a5d1f272720b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:31.823608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b50013-2551-405d-97ac-a7b2636707a3 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.825094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10a74f-9a1d-499c-b66d-a5d1f272720b service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.829479 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca6ab7-6402-49a0-bd8f-59c05620a28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.830248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca6ab7-6402-49a0-bd8f-59c05620a28c admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.830546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca6ab7-6402-49a0-bd8f-59c05620a28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.831087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca6ab7-6402-49a0-bd8f-59c05620a28c admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.832603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca6ab7-6402-49a0-bd8f-59c05620a28c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:31.834007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71424d2-4bba-4896-8e2e-840bdb93ea64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.834344 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca6ab7-6402-49a0-bd8f-59c05620a28c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:31.835827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71424d2-4bba-4896-8e2e-840bdb93ea64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.837781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71424d2-4bba-4896-8e2e-840bdb93ea64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.837781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71424d2-4bba-4896-8e2e-840bdb93ea64 admin admin] PATH_INFO: `/v3/projects/a22190f157ba424d8ae39d7b5705fabe` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.839811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71424d2-4bba-4896-8e2e-840bdb93ea64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a22190f157ba424d8ae39d7b5705fabe)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:31.844958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71424d2-4bba-4896-8e2e-840bdb93ea64 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:31.867514 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1115/2210] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:31 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:31.874318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca6ab7-6402-49a0-bd8f-59c05620a28c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.887404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca6ab7-6402-49a0-bd8f-59c05620a28c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.899671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc7d40-14dd-4d36-b014-12754aa31ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.900685 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc7d40-14dd-4d36-b014-12754aa31ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.901092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc7d40-14dd-4d36-b014-12754aa31ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.901481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc7d40-14dd-4d36-b014-12754aa31ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.903090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc7d40-14dd-4d36-b014-12754aa31ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:31.905260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc7d40-14dd-4d36-b014-12754aa31ae2 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:31.924715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1116/2211] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:31.932876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc7d40-14dd-4d36-b014-12754aa31ae2 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.948540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc7d40-14dd-4d36-b014-12754aa31ae2 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.973714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b71424d2-4bba-4896-8e2e-840bdb93ea64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a22190f157ba424d8ae39d7b5705fabe', 'request_id': 'req-b71424d2-4bba-4896-8e2e-840bdb93ea64'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:31.978674 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:31.995389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b71424d2-4bba-4896-8e2e-840bdb93ea64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a22190f157ba424d8ae39d7b5705fabe', 'request_id': 'req-b71424d2-4bba-4896-8e2e-840bdb93ea64'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:32.017895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b71424d2-4bba-4896-8e2e-840bdb93ea64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a22190f157ba424d8ae39d7b5705fabe', 'request_id': 'req-b71424d2-4bba-4896-8e2e-840bdb93ea64'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:32.028738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dd47c-d776-4025-8157-ffe5a6a186ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.029591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dd47c-d776-4025-8157-ffe5a6a186ca admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.029954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dd47c-d776-4025-8157-ffe5a6a186ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.030331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dd47c-d776-4025-8157-ffe5a6a186ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.031272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dd47c-d776-4025-8157-ffe5a6a186ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.033145 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dd47c-d776-4025-8157-ffe5a6a186ca admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.044968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b71424d2-4bba-4896-8e2e-840bdb93ea64 admin admin] The token cache is being invalidate because project a22190f157ba424d8ae39d7b5705fabe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:32.044968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b71424d2-4bba-4896-8e2e-840bdb93ea64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:32.046289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1117/2212] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:32.048710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b71424d2-4bba-4896-8e2e-840bdb93ea64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:32.051621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1096/2213] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:31 2022] DELETE /identity/v3/projects/a22190f157ba424d8ae39d7b5705fabe => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:32.058227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dd47c-d776-4025-8157-ffe5a6a186ca admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.060839 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-e42dd47c-d776-4025-8157-ffe5a6a186ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:32.061324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7efd9-2468-4d24-ab94-9a44e5bd6150 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.062235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7efd9-2468-4d24-ab94-9a44e5bd6150 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.063117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7efd9-2468-4d24-ab94-9a44e5bd6150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.085764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dd47c-d776-4025-8157-ffe5a6a186ca admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.096271 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f41b1-4829-471a-9b9c-b59104171c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.099068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f41b1-4829-471a-9b9c-b59104171c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.099969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f41b1-4829-471a-9b9c-b59104171c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.100446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f41b1-4829-471a-9b9c-b59104171c2f admin admin] PATH_INFO: `/v3/projects/78240e67d1eb4161a80b2473a50011af/users/b29cfabbbf7d4c28a983bd1c740dbed4/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.112359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7da7efd9-2468-4d24-ab94-9a44e5bd6150 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:32.114866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7da7efd9-2468-4d24-ab94-9a44e5bd6150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:32.132933 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1097/2214] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:32.140417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f41b1-4829-471a-9b9c-b59104171c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78240e67d1eb4161a80b2473a50011af, user_id=b29cfabbbf7d4c28a983bd1c740dbed4, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.143214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c236767-733a-48c9-82a4-6d7bdb393338 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.143453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f41b1-4829-471a-9b9c-b59104171c2f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.143645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c236767-733a-48c9-82a4-6d7bdb393338 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.144072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c236767-733a-48c9-82a4-6d7bdb393338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.161030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1118/2215] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:32 2022] PUT /identity/v3/projects/78240e67d1eb4161a80b2473a50011af/users/b29cfabbbf7d4c28a983bd1c740dbed4/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:32.173797 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec158c-9659-4098-ad2d-ab3d850bccb0 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.174078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec158c-9659-4098-ad2d-ab3d850bccb0 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.174388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec158c-9659-4098-ad2d-ab3d850bccb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.200548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3c236767-733a-48c9-82a4-6d7bdb393338 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:32.203625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3c236767-733a-48c9-82a4-6d7bdb393338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:32.211626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-22ec158c-9659-4098-ad2d-ab3d850bccb0 None None] MFA Rules not processed for user `b29cfabbbf7d4c28a983bd1c740dbed4`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:32.213955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-22ec158c-9659-4098-ad2d-ab3d850bccb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:32.341078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:32.367452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1119/2216] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:32.425638 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1098/2217] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:32.429737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec158c-9659-4098-ad2d-ab3d850bccb0 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.454449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec158c-9659-4098-ad2d-ab3d850bccb0 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.469130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f729a-45cb-4327-ad2a-b2d91d730e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.470133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f729a-45cb-4327-ad2a-b2d91d730e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.470621 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f729a-45cb-4327-ad2a-b2d91d730e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.470927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f729a-45cb-4327-ad2a-b2d91d730e00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.471861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f729a-45cb-4327-ad2a-b2d91d730e00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:32.472208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f729a-45cb-4327-ad2a-b2d91d730e00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.476910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f729a-45cb-4327-ad2a-b2d91d730e00 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.510062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1120/2218] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:32.525026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c236767-733a-48c9-82a4-6d7bdb393338 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.539097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c236767-733a-48c9-82a4-6d7bdb393338 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.545149 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13438da9-95f0-4197-84b9-224881f44b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.545925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13438da9-95f0-4197-84b9-224881f44b20 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.546465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13438da9-95f0-4197-84b9-224881f44b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.546807 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13438da9-95f0-4197-84b9-224881f44b20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.547535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13438da9-95f0-4197-84b9-224881f44b20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.549556 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13438da9-95f0-4197-84b9-224881f44b20 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.603552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1099/2219] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:32 2022] POST /identity/v3/projects => generated 399 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:32.608659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f729a-45cb-4327-ad2a-b2d91d730e00 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.621226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13438da9-95f0-4197-84b9-224881f44b20 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.623276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-13438da9-95f0-4197-84b9-224881f44b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:32.666504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f729a-45cb-4327-ad2a-b2d91d730e00 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.775365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13438da9-95f0-4197-84b9-224881f44b20 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.786413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ab5e2-c38e-45ac-958f-967b3ef4f576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.787381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ab5e2-c38e-45ac-958f-967b3ef4f576 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.787687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ab5e2-c38e-45ac-958f-967b3ef4f576 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.787967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ab5e2-c38e-45ac-958f-967b3ef4f576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.835915 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756bedd-27c5-4f84-8832-8181bcfddceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.836896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756bedd-27c5-4f84-8832-8181bcfddceb admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.837187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756bedd-27c5-4f84-8832-8181bcfddceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.837470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756bedd-27c5-4f84-8832-8181bcfddceb admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.838205 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756bedd-27c5-4f84-8832-8181bcfddceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.841335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756bedd-27c5-4f84-8832-8181bcfddceb admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.861888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:32.878559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1121/2220] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:32 2022] POST /identity/v3/users => generated 300 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:32.882133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756bedd-27c5-4f84-8832-8181bcfddceb admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.882313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ab5e2-c38e-45ac-958f-967b3ef4f576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.885316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ab5e2-c38e-45ac-958f-967b3ef4f576 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.895893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756bedd-27c5-4f84-8832-8181bcfddceb admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.899778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1100/2221] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:32.902463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee8661-e486-4bfb-b5b7-36874a7584bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.903198 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee8661-e486-4bfb-b5b7-36874a7584bd admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.904087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee8661-e486-4bfb-b5b7-36874a7584bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.904087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee8661-e486-4bfb-b5b7-36874a7584bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.904703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee8661-e486-4bfb-b5b7-36874a7584bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.906419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee8661-e486-4bfb-b5b7-36874a7584bd admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.920368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1122/2222] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:32.924653 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ab5e2-c38e-45ac-958f-967b3ef4f576 service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.942784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ab5e2-c38e-45ac-958f-967b3ef4f576 service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.950846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d201f-f4cd-424c-99b4-40e92630596c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.952199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d201f-f4cd-424c-99b4-40e92630596c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.952626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d201f-f4cd-424c-99b4-40e92630596c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.952814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d201f-f4cd-424c-99b4-40e92630596c admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.954347 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d201f-f4cd-424c-99b4-40e92630596c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.956353 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d201f-f4cd-424c-99b4-40e92630596c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.972021 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1101/2223] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:32.976869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee8661-e486-4bfb-b5b7-36874a7584bd admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.998797 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee8661-e486-4bfb-b5b7-36874a7584bd admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.006521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f4e10-0c0c-47b0-b7af-c0585a39699f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.007790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f4e10-0c0c-47b0-b7af-c0585a39699f admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.008127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f4e10-0c0c-47b0-b7af-c0585a39699f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.010192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f4e10-0c0c-47b0-b7af-c0585a39699f admin admin] PATH_INFO: `/v3/projects/d380d7b17929495a93f61991756c6467/users/2ea7e7f7373a4f648392809c2e70fb6a/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.032567 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f4e10-0c0c-47b0-b7af-c0585a39699f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d380d7b17929495a93f61991756c6467, user_id=2ea7e7f7373a4f648392809c2e70fb6a, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.038114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f4e10-0c0c-47b0-b7af-c0585a39699f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.060850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1123/2224] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:32 2022] PUT /identity/v3/projects/d380d7b17929495a93f61991756c6467/users/2ea7e7f7373a4f648392809c2e70fb6a/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:33.074494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0adb40-0236-4ea7-9edb-828bdfd52e13 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.075324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0adb40-0236-4ea7-9edb-828bdfd52e13 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.075324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0adb40-0236-4ea7-9edb-828bdfd52e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.130161 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-be0adb40-0236-4ea7-9edb-828bdfd52e13 None None] MFA Rules not processed for user `2ea7e7f7373a4f648392809c2e70fb6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:33.131449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-be0adb40-0236-4ea7-9edb-828bdfd52e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:33.249363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1102/2225] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:33.253815 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f4e10-0c0c-47b0-b7af-c0585a39699f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.391332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f4e10-0c0c-47b0-b7af-c0585a39699f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.408849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96ccb1-7ec4-4eff-86cc-457e9509c0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.408849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96ccb1-7ec4-4eff-86cc-457e9509c0f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.408849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96ccb1-7ec4-4eff-86cc-457e9509c0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.408849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96ccb1-7ec4-4eff-86cc-457e9509c0f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.410794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96ccb1-7ec4-4eff-86cc-457e9509c0f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.410794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96ccb1-7ec4-4eff-86cc-457e9509c0f6 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.459322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1124/2226] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:33 2022] POST /identity/v3/projects => generated 401 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:33.465705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0adb40-0236-4ea7-9edb-828bdfd52e13 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.478776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0adb40-0236-4ea7-9edb-828bdfd52e13 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.579767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157a48e-95ed-4c40-9d0a-135d06cc3bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.580471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157a48e-95ed-4c40-9d0a-135d06cc3bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.580865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157a48e-95ed-4c40-9d0a-135d06cc3bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.581222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157a48e-95ed-4c40-9d0a-135d06cc3bd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.582010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157a48e-95ed-4c40-9d0a-135d06cc3bd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.584194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157a48e-95ed-4c40-9d0a-135d06cc3bd8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.621699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1103/2227] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:33 2022] POST /identity/v3/users => generated 307 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:33.628038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96ccb1-7ec4-4eff-86cc-457e9509c0f6 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.646190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96ccb1-7ec4-4eff-86cc-457e9509c0f6 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.658589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf64bc5-7e9e-42d7-8e35-ad728db2b18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.662029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf64bc5-7e9e-42d7-8e35-ad728db2b18a admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.662762 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf64bc5-7e9e-42d7-8e35-ad728db2b18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.663259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf64bc5-7e9e-42d7-8e35-ad728db2b18a admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.664869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf64bc5-7e9e-42d7-8e35-ad728db2b18a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.668665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf64bc5-7e9e-42d7-8e35-ad728db2b18a admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.692069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1125/2228] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:33.700663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157a48e-95ed-4c40-9d0a-135d06cc3bd8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.721732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157a48e-95ed-4c40-9d0a-135d06cc3bd8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.730237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffecb0-51e8-4624-88cf-1e039a42b05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.731802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffecb0-51e8-4624-88cf-1e039a42b05e admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.732273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffecb0-51e8-4624-88cf-1e039a42b05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.732791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffecb0-51e8-4624-88cf-1e039a42b05e admin admin] PATH_INFO: `/v3/projects/477806f8531a460ba5edf523bff0e65b/users/09e6548d18df45868a632598ac300b53/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.753815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffecb0-51e8-4624-88cf-1e039a42b05e admin admin] RBAC: Authorizing `identity:create_grant(project_id=477806f8531a460ba5edf523bff0e65b, user_id=09e6548d18df45868a632598ac300b53, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.757253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffecb0-51e8-4624-88cf-1e039a42b05e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.777869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1104/2229] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:33 2022] PUT /identity/v3/projects/477806f8531a460ba5edf523bff0e65b/users/09e6548d18df45868a632598ac300b53/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:33.784090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf64bc5-7e9e-42d7-8e35-ad728db2b18a admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.803539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf64bc5-7e9e-42d7-8e35-ad728db2b18a admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.818231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:33.934598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0bbcc-2f01-495d-87f2-2e5fcbcbf10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.936589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0bbcc-2f01-495d-87f2-2e5fcbcbf10e admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.937062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0bbcc-2f01-495d-87f2-2e5fcbcbf10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.937629 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0bbcc-2f01-495d-87f2-2e5fcbcbf10e admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.938922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0bbcc-2f01-495d-87f2-2e5fcbcbf10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.944114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0bbcc-2f01-495d-87f2-2e5fcbcbf10e admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.976735 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1126/2230] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:33.978796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffecb0-51e8-4624-88cf-1e039a42b05e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.018678 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffecb0-51e8-4624-88cf-1e039a42b05e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:34.030142 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb2388-df5d-4e3a-ade6-1b483a58352e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:34.031799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb2388-df5d-4e3a-ade6-1b483a58352e admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.032274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb2388-df5d-4e3a-ade6-1b483a58352e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.033180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb2388-df5d-4e3a-ade6-1b483a58352e admin admin] PATH_INFO: `/v3/projects/477806f8531a460ba5edf523bff0e65b/users/09e6548d18df45868a632598ac300b53/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.040530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb2388-df5d-4e3a-ade6-1b483a58352e admin admin] RBAC: Authorizing `identity:create_grant(project_id=477806f8531a460ba5edf523bff0e65b, user_id=09e6548d18df45868a632598ac300b53, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:34.047099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb2388-df5d-4e3a-ade6-1b483a58352e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:34.084544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1105/2231] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:33 2022] PUT /identity/v3/projects/477806f8531a460ba5edf523bff0e65b/users/09e6548d18df45868a632598ac300b53/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:34.105280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e287a-7b64-4e32-8e2a-14e7023c6f3b None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.105806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e287a-7b64-4e32-8e2a-14e7023c6f3b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.106318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e287a-7b64-4e32-8e2a-14e7023c6f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.188277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-cb1e287a-7b64-4e32-8e2a-14e7023c6f3b None None] MFA Rules not processed for user `09e6548d18df45868a632598ac300b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:34.191313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-cb1e287a-7b64-4e32-8e2a-14e7023c6f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:34.300741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1127/2232] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:34 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:34.314634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb2388-df5d-4e3a-ade6-1b483a58352e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.316967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b8fb2388-df5d-4e3a-ade6-1b483a58352e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:34.342437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:34.388764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb2388-df5d-4e3a-ade6-1b483a58352e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:34.398010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830d40f-d144-4b52-b143-143b2c74bb37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:34.398308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830d40f-d144-4b52-b143-143b2c74bb37 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.398484 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830d40f-d144-4b52-b143-143b2c74bb37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.398696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830d40f-d144-4b52-b143-143b2c74bb37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.413288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830d40f-d144-4b52-b143-143b2c74bb37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:34.415992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830d40f-d144-4b52-b143-143b2c74bb37 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:34.428121 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1106/2233] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:34 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:34.522274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e287a-7b64-4e32-8e2a-14e7023c6f3b None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.533947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e287a-7b64-4e32-8e2a-14e7023c6f3b None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:34.540914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf2ff5-ee3d-4525-ac57-aa379a15a932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:34.541905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf2ff5-ee3d-4525-ac57-aa379a15a932 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.542843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf2ff5-ee3d-4525-ac57-aa379a15a932 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.543551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf2ff5-ee3d-4525-ac57-aa379a15a932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.643690 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf2ff5-ee3d-4525-ac57-aa379a15a932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:34.645728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf2ff5-ee3d-4525-ac57-aa379a15a932 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:34.666810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1128/2234] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:38.875995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830d40f-d144-4b52-b143-143b2c74bb37 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:38.877515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6830d40f-d144-4b52-b143-143b2c74bb37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:38.890129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830d40f-d144-4b52-b143-143b2c74bb37 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:38.893850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58284359-756f-4e63-9d5b-b12cc46c32c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:38.894500 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58284359-756f-4e63-9d5b-b12cc46c32c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:38.894782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58284359-756f-4e63-9d5b-b12cc46c32c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:38.895056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58284359-756f-4e63-9d5b-b12cc46c32c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:38.897088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-58284359-756f-4e63-9d5b-b12cc46c32c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:38.966073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58284359-756f-4e63-9d5b-b12cc46c32c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:38.967522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58284359-756f-4e63-9d5b-b12cc46c32c8 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:38.978473 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1107/2235] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:41.979264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf2ff5-ee3d-4525-ac57-aa379a15a932 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:42.050228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:42.105107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf2ff5-ee3d-4525-ac57-aa379a15a932 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:42.112736 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:42.113071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.113409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.113697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] PATH_INFO: `/v3/users/b29cfabbbf7d4c28a983bd1c740dbed4` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.115194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] RBAC: Authorizing `identity:delete_user(user_id=b29cfabbbf7d4c28a983bd1c740dbed4)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:42.117970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:42.184173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b29cfabbbf7d4c28a983bd1c740dbed4'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:42.201874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b29cfabbbf7d4c28a983bd1c740dbed4'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:42.202757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b29cfabbbf7d4c28a983bd1c740dbed4'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:42.217162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b29cfabbbf7d4c28a983bd1c740dbed4'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:42.218260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b29cfabbbf7d4c28a983bd1c740dbed4'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:42.242954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1129/2236] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:41 2022] DELETE /identity/v3/users/b29cfabbbf7d4c28a983bd1c740dbed4 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:42.248807 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1967f-9ed4-40e2-80bd-016a5750a783 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.249406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1967f-9ed4-40e2-80bd-016a5750a783 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.249406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1967f-9ed4-40e2-80bd-016a5750a783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.271270 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3bb1967f-9ed4-40e2-80bd-016a5750a783 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:42.272581 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3bb1967f-9ed4-40e2-80bd-016a5750a783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:42.388094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1108/2237] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:42.400387 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:42.401981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:42.551318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde37933-9b90-4477-8e4e-c0cea2b8b6be admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:42.559099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309d716-6dc6-4aad-a617-73a498c11053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:42.560150 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309d716-6dc6-4aad-a617-73a498c11053 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.560524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309d716-6dc6-4aad-a617-73a498c11053 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.561603 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309d716-6dc6-4aad-a617-73a498c11053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.575710 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309d716-6dc6-4aad-a617-73a498c11053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:42.577651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309d716-6dc6-4aad-a617-73a498c11053 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:42.598837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1130/2238] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:42.665592 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1967f-9ed4-40e2-80bd-016a5750a783 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:42.667505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3bb1967f-9ed4-40e2-80bd-016a5750a783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:42.691740 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1967f-9ed4-40e2-80bd-016a5750a783 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:42.700909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:42.702361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.702854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.703552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] PATH_INFO: `/v3/users/2ea7e7f7373a4f648392809c2e70fb6a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.705695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea7e7f7373a4f648392809c2e70fb6a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:42.710369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:42.792260 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea7e7f7373a4f648392809c2e70fb6a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:42.794406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:42.802162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea7e7f7373a4f648392809c2e70fb6a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:42.819452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea7e7f7373a4f648392809c2e70fb6a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:42.832050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea7e7f7373a4f648392809c2e70fb6a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:42.832917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea7e7f7373a4f648392809c2e70fb6a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:42.841595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1109/2239] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:42 2022] DELETE /identity/v3/users/2ea7e7f7373a4f648392809c2e70fb6a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:42.847697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669eb97e-d8b0-4a7f-b4bb-271efa0805a8 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.848035 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669eb97e-d8b0-4a7f-b4bb-271efa0805a8 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.848325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669eb97e-d8b0-4a7f-b4bb-271efa0805a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.868641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-669eb97e-d8b0-4a7f-b4bb-271efa0805a8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:42.869956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-669eb97e-d8b0-4a7f-b4bb-271efa0805a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:42.949868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:42.971736 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1131/2240] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:43.008776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.010207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:43.055924 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669eb97e-d8b0-4a7f-b4bb-271efa0805a8 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.058346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-669eb97e-d8b0-4a7f-b4bb-271efa0805a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:43.073951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669eb97e-d8b0-4a7f-b4bb-271efa0805a8 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.079483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.080923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.081654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.082090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8 admin admin] PATH_INFO: `/v3/projects/78240e67d1eb4161a80b2473a50011af` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.083714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78240e67d1eb4161a80b2473a50011af)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.086482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.123072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c1789-00cc-4929-a4ee-e5f84bdde1a0 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.131984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466c2f3-856f-48ea-bbc8-4b180f736e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.134280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466c2f3-856f-48ea-bbc8-4b180f736e09 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.134568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466c2f3-856f-48ea-bbc8-4b180f736e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.134929 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466c2f3-856f-48ea-bbc8-4b180f736e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.149839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466c2f3-856f-48ea-bbc8-4b180f736e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.151322 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5466c2f3-856f-48ea-bbc8-4b180f736e09 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.163528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1110/2241] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:43.167521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78240e67d1eb4161a80b2473a50011af', 'request_id': 'req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:43.177363 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78240e67d1eb4161a80b2473a50011af', 'request_id': 'req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:43.190574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78240e67d1eb4161a80b2473a50011af', 'request_id': 'req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:43.213031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8 admin admin] The token cache is being invalidate because project 78240e67d1eb4161a80b2473a50011af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:43.213459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:43.215529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-45c5b235-a8c1-4789-97ba-9a60c9a0d6c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:43.219295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1132/2242] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:43 2022] DELETE /identity/v3/projects/78240e67d1eb4161a80b2473a50011af => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:43.229175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44017500-a34b-44c7-b923-80198b164f80 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.229654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44017500-a34b-44c7-b923-80198b164f80 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.230115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44017500-a34b-44c7-b923-80198b164f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.254829 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-44017500-a34b-44c7-b923-80198b164f80 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:43.257016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-44017500-a34b-44c7-b923-80198b164f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:43.357861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1111/2243] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:43.367337 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afb731-2df8-4e7e-88ff-ab750d2fd4ca None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.367953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afb731-2df8-4e7e-88ff-ab750d2fd4ca None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.368424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afb731-2df8-4e7e-88ff-ab750d2fd4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.396264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-96afb731-2df8-4e7e-88ff-ab750d2fd4ca None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:43.399135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-96afb731-2df8-4e7e-88ff-ab750d2fd4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:43.412982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1133/2244] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:43.419620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44017500-a34b-44c7-b923-80198b164f80 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.433407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44017500-a34b-44c7-b923-80198b164f80 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.445114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54702660-0844-42f3-bc6c-5a1a6e9c974d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.446741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54702660-0844-42f3-bc6c-5a1a6e9c974d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.447383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54702660-0844-42f3-bc6c-5a1a6e9c974d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.447847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54702660-0844-42f3-bc6c-5a1a6e9c974d admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.449153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54702660-0844-42f3-bc6c-5a1a6e9c974d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:43.449604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54702660-0844-42f3-bc6c-5a1a6e9c974d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.451562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54702660-0844-42f3-bc6c-5a1a6e9c974d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.466540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1112/2245] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:43.472903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afb731-2df8-4e7e-88ff-ab750d2fd4ca None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.487724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afb731-2df8-4e7e-88ff-ab750d2fd4ca None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.495912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9744293-8465-4abd-99e5-511275cf217c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.496681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9744293-8465-4abd-99e5-511275cf217c admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.496954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9744293-8465-4abd-99e5-511275cf217c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.497240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9744293-8465-4abd-99e5-511275cf217c admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.497974 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9744293-8465-4abd-99e5-511275cf217c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.499958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9744293-8465-4abd-99e5-511275cf217c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.535558 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1134/2246] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:43 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:43.537666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54702660-0844-42f3-bc6c-5a1a6e9c974d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.558199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54702660-0844-42f3-bc6c-5a1a6e9c974d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.658574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9744293-8465-4abd-99e5-511275cf217c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.661729 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f9744293-8465-4abd-99e5-511275cf217c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:43.671458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:43.686388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c0c0d-71cd-473e-a10e-650fddfb1109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.687989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c0c0d-71cd-473e-a10e-650fddfb1109 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.688840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c0c0d-71cd-473e-a10e-650fddfb1109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.689495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c0c0d-71cd-473e-a10e-650fddfb1109 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.692380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c0c0d-71cd-473e-a10e-650fddfb1109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.704879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c0c0d-71cd-473e-a10e-650fddfb1109 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.753690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1113/2247] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:43 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:43.754895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c0c0d-71cd-473e-a10e-650fddfb1109 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.771540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c0c0d-71cd-473e-a10e-650fddfb1109 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.784506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51c99e-8ed8-431a-9ce2-3c9745469347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.785213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51c99e-8ed8-431a-9ce2-3c9745469347 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.785504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51c99e-8ed8-431a-9ce2-3c9745469347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.785787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51c99e-8ed8-431a-9ce2-3c9745469347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.786703 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51c99e-8ed8-431a-9ce2-3c9745469347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.788211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51c99e-8ed8-431a-9ce2-3c9745469347 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.799624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9744293-8465-4abd-99e5-511275cf217c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.802431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1114/2248] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:43.807394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51c99e-8ed8-431a-9ce2-3c9745469347 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.816153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.817308 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.817675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.818017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] PATH_INFO: `/v3/projects/d380d7b17929495a93f61991756c6467` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.819469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d380d7b17929495a93f61991756c6467)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.821863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.823172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51c99e-8ed8-431a-9ce2-3c9745469347 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.829522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6d2ee-498a-4365-9b36-2afdac06e95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.830868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6d2ee-498a-4365-9b36-2afdac06e95d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.831414 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6d2ee-498a-4365-9b36-2afdac06e95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.831771 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6d2ee-498a-4365-9b36-2afdac06e95d admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.833295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6d2ee-498a-4365-9b36-2afdac06e95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.835581 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6d2ee-498a-4365-9b36-2afdac06e95d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.859632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1115/2249] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:43.863593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6d2ee-498a-4365-9b36-2afdac06e95d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.888236 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6d2ee-498a-4365-9b36-2afdac06e95d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.901731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd380d7b17929495a93f61991756c6467', 'request_id': 'req-e3345dc9-750d-42a3-9977-a64ce3430ba6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:43.904322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:43.920857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd380d7b17929495a93f61991756c6467', 'request_id': 'req-e3345dc9-750d-42a3-9977-a64ce3430ba6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:43.934764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd380d7b17929495a93f61991756c6467', 'request_id': 'req-e3345dc9-750d-42a3-9977-a64ce3430ba6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:43.968889 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] The token cache is being invalidate because project d380d7b17929495a93f61991756c6467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:43.969731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:43.971516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:43.980566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1135/2250] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:43 2022] DELETE /identity/v3/projects/d380d7b17929495a93f61991756c6467 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:43.982411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.985078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:44.030562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98d8f4-e0a4-48a2-92a3-49b49879bfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.031669 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98d8f4-e0a4-48a2-92a3-49b49879bfca admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.032021 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98d8f4-e0a4-48a2-92a3-49b49879bfca admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.032363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98d8f4-e0a4-48a2-92a3-49b49879bfca admin admin] PATH_INFO: `/v3/projects/5360faa4319b445cbfbb3bc3581e0e1d/users/e3c3383707e34f3c9d3707b52143202b/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.057854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98d8f4-e0a4-48a2-92a3-49b49879bfca admin admin] RBAC: Authorizing `identity:create_grant(project_id=5360faa4319b445cbfbb3bc3581e0e1d, user_id=e3c3383707e34f3c9d3707b52143202b, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.061038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98d8f4-e0a4-48a2-92a3-49b49879bfca admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.081664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1116/2251] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:43 2022] PUT /identity/v3/projects/5360faa4319b445cbfbb3bc3581e0e1d/users/e3c3383707e34f3c9d3707b52143202b/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:44.093631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dba96-785a-4f12-a143-56638548ca28 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.094049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dba96-785a-4f12-a143-56638548ca28 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.094427 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dba96-785a-4f12-a143-56638548ca28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.119907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3345dc9-750d-42a3-9977-a64ce3430ba6 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.134886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5c6dba96-785a-4f12-a143-56638548ca28 None None] MFA Rules not processed for user `e3c3383707e34f3c9d3707b52143202b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:44.136203 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5c6dba96-785a-4f12-a143-56638548ca28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:44.189194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.190085 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.190376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.190656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] PATH_INFO: `/v3/users/09e6548d18df45868a632598ac300b53` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.192563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e6548d18df45868a632598ac300b53)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.198105 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.250788 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1117/2252] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:44.255057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dba96-785a-4f12-a143-56638548ca28 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.256862 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5c6dba96-785a-4f12-a143-56638548ca28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:44.278986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dba96-785a-4f12-a143-56638548ca28 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.287747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd62f9a-f770-4763-82a3-e05619c5a579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.288732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd62f9a-f770-4763-82a3-e05619c5a579 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.288973 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd62f9a-f770-4763-82a3-e05619c5a579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.289260 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd62f9a-f770-4763-82a3-e05619c5a579 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.289976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd62f9a-f770-4763-82a3-e05619c5a579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.293118 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd62f9a-f770-4763-82a3-e05619c5a579 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.302714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e6548d18df45868a632598ac300b53'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:44.315017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e6548d18df45868a632598ac300b53'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:44.316173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e6548d18df45868a632598ac300b53'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:44.332293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1118/2253] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:44 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:44.332293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd62f9a-f770-4763-82a3-e05619c5a579 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.332293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ffd62f9a-f770-4763-82a3-e05619c5a579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:44.336260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e6548d18df45868a632598ac300b53'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:44.337571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e6548d18df45868a632598ac300b53'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:44.364106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:44.372115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1136/2254] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:43 2022] DELETE /identity/v3/users/09e6548d18df45868a632598ac300b53 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:44.372115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.372115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:44.378652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:44.457245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd62f9a-f770-4763-82a3-e05619c5a579 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.509101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2edcf-9c23-4bd2-b31c-e775a6bec4b3 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.518937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a6961-1274-48d6-a069-d295679bd1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.520506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a6961-1274-48d6-a069-d295679bd1e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.522486 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a6961-1274-48d6-a069-d295679bd1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.522486 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a6961-1274-48d6-a069-d295679bd1e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.523099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a6961-1274-48d6-a069-d295679bd1e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.526903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a6961-1274-48d6-a069-d295679bd1e6 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.580136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be69d5-f4c4-4a1a-82c0-5b3a7d703b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.580679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1137/2255] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:44 2022] POST /identity/v3/users => generated 290 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:44.581666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be69d5-f4c4-4a1a-82c0-5b3a7d703b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.582024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be69d5-f4c4-4a1a-82c0-5b3a7d703b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.582368 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be69d5-f4c4-4a1a-82c0-5b3a7d703b47 admin admin] PATH_INFO: `/v3/users/2ba346eef71c47a7819896c8cc6a02fe` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.584354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be69d5-f4c4-4a1a-82c0-5b3a7d703b47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ba346eef71c47a7819896c8cc6a02fe)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.586416 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a6961-1274-48d6-a069-d295679bd1e6 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.586698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be69d5-f4c4-4a1a-82c0-5b3a7d703b47 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.609178 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a6961-1274-48d6-a069-d295679bd1e6 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.618901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f234920-0684-420c-9dbc-4d753af015b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.620402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f234920-0684-420c-9dbc-4d753af015b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.621225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f234920-0684-420c-9dbc-4d753af015b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.621960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f234920-0684-420c-9dbc-4d753af015b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.623723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f234920-0684-420c-9dbc-4d753af015b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.626790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f234920-0684-420c-9dbc-4d753af015b1 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.650171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1138/2256] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:44.661998 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f234920-0684-420c-9dbc-4d753af015b1 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.682154 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f234920-0684-420c-9dbc-4d753af015b1 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.687978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb3ddd-1acc-4b48-86be-63306cd5b629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.688730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb3ddd-1acc-4b48-86be-63306cd5b629 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.689028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb3ddd-1acc-4b48-86be-63306cd5b629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.689313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb3ddd-1acc-4b48-86be-63306cd5b629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.690084 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb3ddd-1acc-4b48-86be-63306cd5b629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.691318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-73be69d5-f4c4-4a1a-82c0-5b3a7d703b47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ba346eef71c47a7819896c8cc6a02fe'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:44.692144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfb3ddd-1acc-4b48-86be-63306cd5b629 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.705079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1139/2257] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:44.709777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb3ddd-1acc-4b48-86be-63306cd5b629 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.713544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-73be69d5-f4c4-4a1a-82c0-5b3a7d703b47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ba346eef71c47a7819896c8cc6a02fe'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:44.721468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb3ddd-1acc-4b48-86be-63306cd5b629 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.729081 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd3476-28cb-409a-91fc-4d39a6423470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.730278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd3476-28cb-409a-91fc-4d39a6423470 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.730666 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd3476-28cb-409a-91fc-4d39a6423470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.730997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd3476-28cb-409a-91fc-4d39a6423470 admin admin] PATH_INFO: `/v3/projects/7d28f1b462824d96ad689a5804d08771/users/f9d512bc874949ceaded0682d0f68e90/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.739617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-73be69d5-f4c4-4a1a-82c0-5b3a7d703b47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ba346eef71c47a7819896c8cc6a02fe'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:44.751661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd3476-28cb-409a-91fc-4d39a6423470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d28f1b462824d96ad689a5804d08771, user_id=f9d512bc874949ceaded0682d0f68e90, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.756004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd3476-28cb-409a-91fc-4d39a6423470 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.757197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-73be69d5-f4c4-4a1a-82c0-5b3a7d703b47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ba346eef71c47a7819896c8cc6a02fe'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:44.758350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-73be69d5-f4c4-4a1a-82c0-5b3a7d703b47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ba346eef71c47a7819896c8cc6a02fe'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:44.780550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1119/2259] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:44 2022] DELETE /identity/v3/users/2ba346eef71c47a7819896c8cc6a02fe => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:44.787021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1140/2259] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:44 2022] PUT /identity/v3/projects/7d28f1b462824d96ad689a5804d08771/users/f9d512bc874949ceaded0682d0f68e90/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:44.789767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50ffa1-6476-433d-afe4-340937051f22 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.790148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50ffa1-6476-433d-afe4-340937051f22 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.790442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50ffa1-6476-433d-afe4-340937051f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.791878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00d476-780f-4a9f-a002-fc886b692839 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.792380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00d476-780f-4a9f-a002-fc886b692839 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.792889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00d476-780f-4a9f-a002-fc886b692839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.819716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-fd00d476-780f-4a9f-a002-fc886b692839 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:44.822367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fd00d476-780f-4a9f-a002-fc886b692839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:44.835811 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ce50ffa1-6476-433d-afe4-340937051f22 None None] MFA Rules not processed for user `f9d512bc874949ceaded0682d0f68e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:44.839720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ce50ffa1-6476-433d-afe4-340937051f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:44.881039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:44.918958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1120/2260] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:44.920716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00d476-780f-4a9f-a002-fc886b692839 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.935890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fd00d476-780f-4a9f-a002-fc886b692839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:44.961966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00d476-780f-4a9f-a002-fc886b692839 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.970459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.972589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.973126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.973624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] PATH_INFO: `/v3/projects/477806f8531a460ba5edf523bff0e65b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.975348 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=477806f8531a460ba5edf523bff0e65b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.981010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.987956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1141/2261] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:44.996541 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50ffa1-6476-433d-afe4-340937051f22 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.999116 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ce50ffa1-6476-433d-afe4-340937051f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:45.065708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '477806f8531a460ba5edf523bff0e65b', 'request_id': 'req-804e0398-3b76-4759-bffb-fd5fb8642518'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:45.075927 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '477806f8531a460ba5edf523bff0e65b', 'request_id': 'req-804e0398-3b76-4759-bffb-fd5fb8642518'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:45.085840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '477806f8531a460ba5edf523bff0e65b', 'request_id': 'req-804e0398-3b76-4759-bffb-fd5fb8642518'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:45.093646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:45.104148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] The token cache is being invalidate because project 477806f8531a460ba5edf523bff0e65b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:45.104567 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:45.105379 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:45.108753 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1121/2262] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:44 2022] DELETE /identity/v3/projects/477806f8531a460ba5edf523bff0e65b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:45.110030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.111494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:45.138119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50ffa1-6476-433d-afe4-340937051f22 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.233415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5be7b5-1e76-447d-b2f4-217c6ee70a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.234098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5be7b5-1e76-447d-b2f4-217c6ee70a42 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.234398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5be7b5-1e76-447d-b2f4-217c6ee70a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.236585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5be7b5-1e76-447d-b2f4-217c6ee70a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.238989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-dd5be7b5-1e76-447d-b2f4-217c6ee70a42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:45.262756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e0398-3b76-4759-bffb-fd5fb8642518 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.269491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4484f-7377-46d2-a9c3-8693c32188cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.270187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4484f-7377-46d2-a9c3-8693c32188cb admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.271217 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4484f-7377-46d2-a9c3-8693c32188cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.271514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4484f-7377-46d2-a9c3-8693c32188cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.272253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4484f-7377-46d2-a9c3-8693c32188cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.275235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4484f-7377-46d2-a9c3-8693c32188cb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.336126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1122/2263] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:45 2022] POST /identity/v3/projects => generated 379 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:45.338304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b1fd9-52ac-4012-b4bb-19f05b6f4a41 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.339286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b1fd9-52ac-4012-b4bb-19f05b6f4a41 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.339722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b1fd9-52ac-4012-b4bb-19f05b6f4a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.344269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5be7b5-1e76-447d-b2f4-217c6ee70a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.346709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5be7b5-1e76-447d-b2f4-217c6ee70a42 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.360947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1142/2264] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:45.363292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-bc1b1fd9-52ac-4012-b4bb-19f05b6f4a41 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:45.364707 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bc1b1fd9-52ac-4012-b4bb-19f05b6f4a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:45.366697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5be7b5-1e76-447d-b2f4-217c6ee70a42 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.383352 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5be7b5-1e76-447d-b2f4-217c6ee70a42 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.468208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:45.480309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d163e5a-cd81-44e0-b818-c0683e58a5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.481426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d163e5a-cd81-44e0-b818-c0683e58a5a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.481867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d163e5a-cd81-44e0-b818-c0683e58a5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.482426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d163e5a-cd81-44e0-b818-c0683e58a5a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.485191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d163e5a-cd81-44e0-b818-c0683e58a5a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.488144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d163e5a-cd81-44e0-b818-c0683e58a5a5 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.492815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1123/2265] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:45.501107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913ac4a-e793-4c89-8a8d-ac5e13c82eea None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.501478 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913ac4a-e793-4c89-8a8d-ac5e13c82eea None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.501863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913ac4a-e793-4c89-8a8d-ac5e13c82eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.525975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d913ac4a-e793-4c89-8a8d-ac5e13c82eea None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:45.527247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d913ac4a-e793-4c89-8a8d-ac5e13c82eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:45.542184 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1124/2267] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:45.542529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1143/2267] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:45 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:45.542665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913ac4a-e793-4c89-8a8d-ac5e13c82eea None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.543937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d163e5a-cd81-44e0-b818-c0683e58a5a5 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.554820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913ac4a-e793-4c89-8a8d-ac5e13c82eea None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.560617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d163e5a-cd81-44e0-b818-c0683e58a5a5 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.561123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c33934-aeb6-4454-93d9-f00d3c4c71e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.562426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c33934-aeb6-4454-93d9-f00d3c4c71e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.562426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c33934-aeb6-4454-93d9-f00d3c4c71e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.562426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c33934-aeb6-4454-93d9-f00d3c4c71e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.563288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c33934-aeb6-4454-93d9-f00d3c4c71e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.566670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c33934-aeb6-4454-93d9-f00d3c4c71e8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.574178 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed3d2b-5c48-402d-83f6-1abd6479837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.575112 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3d2b-5c48-402d-83f6-1abd6479837f admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.575562 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3d2b-5c48-402d-83f6-1abd6479837f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.576041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3d2b-5c48-402d-83f6-1abd6479837f admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.577452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed3d2b-5c48-402d-83f6-1abd6479837f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:45.578869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed3d2b-5c48-402d-83f6-1abd6479837f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.581888 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed3d2b-5c48-402d-83f6-1abd6479837f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.591721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1125/2268] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:45.598497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c33934-aeb6-4454-93d9-f00d3c4c71e8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.620491 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1144/2269] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:45.621462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c33934-aeb6-4454-93d9-f00d3c4c71e8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.624125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed3d2b-5c48-402d-83f6-1abd6479837f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.639860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7714568-a17c-4964-af4f-13c8bc315918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.641992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7714568-a17c-4964-af4f-13c8bc315918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.643162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7714568-a17c-4964-af4f-13c8bc315918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.643606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7714568-a17c-4964-af4f-13c8bc315918 admin admin] PATH_INFO: `/v3/projects/b5bac759b6de4bac8faaf7afc3a2621b/users/0e4e0ada75c34457b407a32034ba7aa8/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.644965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed3d2b-5c48-402d-83f6-1abd6479837f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.655769 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952a76d-2a18-4ac6-a5bb-3e3c5e1e4a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.657331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952a76d-2a18-4ac6-a5bb-3e3c5e1e4a95 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.658000 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952a76d-2a18-4ac6-a5bb-3e3c5e1e4a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.658666 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952a76d-2a18-4ac6-a5bb-3e3c5e1e4a95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.660630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952a76d-2a18-4ac6-a5bb-3e3c5e1e4a95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.666294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952a76d-2a18-4ac6-a5bb-3e3c5e1e4a95 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.673541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7714568-a17c-4964-af4f-13c8bc315918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bac759b6de4bac8faaf7afc3a2621b, user_id=0e4e0ada75c34457b407a32034ba7aa8, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.677498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7714568-a17c-4964-af4f-13c8bc315918 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.699246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1126/2270] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:45 2022] PUT /identity/v3/projects/b5bac759b6de4bac8faaf7afc3a2621b/users/0e4e0ada75c34457b407a32034ba7aa8/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:45.704002 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7714568-a17c-4964-af4f-13c8bc315918 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.708931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:45.731731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1145/2271] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:45 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:45.733445 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952a76d-2a18-4ac6-a5bb-3e3c5e1e4a95 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.800470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7714568-a17c-4964-af4f-13c8bc315918 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.852629 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952a76d-2a18-4ac6-a5bb-3e3c5e1e4a95 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.857774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdba4d-e58e-4f0f-aacc-f711f8306bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.858485 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdba4d-e58e-4f0f-aacc-f711f8306bef admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.859097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdba4d-e58e-4f0f-aacc-f711f8306bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.859097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdba4d-e58e-4f0f-aacc-f711f8306bef admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.859769 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdba4d-e58e-4f0f-aacc-f711f8306bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.861925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdba4d-e58e-4f0f-aacc-f711f8306bef admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.883460 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108996e-4b17-4637-af5e-962ed425f3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.884127 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108996e-4b17-4637-af5e-962ed425f3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.884620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108996e-4b17-4637-af5e-962ed425f3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.884999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108996e-4b17-4637-af5e-962ed425f3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.885967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108996e-4b17-4637-af5e-962ed425f3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.888536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108996e-4b17-4637-af5e-962ed425f3a0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.889918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1146/2272] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:45 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:45.893956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdba4d-e58e-4f0f-aacc-f711f8306bef admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.905882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1127/2273] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:45.907262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108996e-4b17-4637-af5e-962ed425f3a0 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.912385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdba4d-e58e-4f0f-aacc-f711f8306bef admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.926223 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e77831-6be7-4174-b25b-81fccbfc5337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.928351 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e77831-6be7-4174-b25b-81fccbfc5337 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.928967 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e77831-6be7-4174-b25b-81fccbfc5337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.930907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e77831-6be7-4174-b25b-81fccbfc5337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.933155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e77831-6be7-4174-b25b-81fccbfc5337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.935652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108996e-4b17-4637-af5e-962ed425f3a0 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.937008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e77831-6be7-4174-b25b-81fccbfc5337 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.943867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588b42f-b475-4379-b569-3a432b8a712c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.947328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588b42f-b475-4379-b569-3a432b8a712c admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.947794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588b42f-b475-4379-b569-3a432b8a712c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.948297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588b42f-b475-4379-b569-3a432b8a712c admin admin] PATH_INFO: `/v3/projects/b5bac759b6de4bac8faaf7afc3a2621b/users/0e4e0ada75c34457b407a32034ba7aa8/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.953225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588b42f-b475-4379-b569-3a432b8a712c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bac759b6de4bac8faaf7afc3a2621b, user_id=0e4e0ada75c34457b407a32034ba7aa8, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.958663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588b42f-b475-4379-b569-3a432b8a712c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.966749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1147/2274] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:45.968458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e77831-6be7-4174-b25b-81fccbfc5337 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.971701 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c8e77831-6be7-4174-b25b-81fccbfc5337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:45.991687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1128/2275] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:45 2022] PUT /identity/v3/projects/b5bac759b6de4bac8faaf7afc3a2621b/users/0e4e0ada75c34457b407a32034ba7aa8/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:45.997497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588b42f-b475-4379-b569-3a432b8a712c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:46.018030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e77831-6be7-4174-b25b-81fccbfc5337 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.051447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:46.076728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588b42f-b475-4379-b569-3a432b8a712c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.082958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd315a-6b44-4cb4-aee0-8c375e808dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.083823 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd315a-6b44-4cb4-aee0-8c375e808dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.084210 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd315a-6b44-4cb4-aee0-8c375e808dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.084724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd315a-6b44-4cb4-aee0-8c375e808dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.085966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd315a-6b44-4cb4-aee0-8c375e808dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.087708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd315a-6b44-4cb4-aee0-8c375e808dc0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.104548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1129/2276] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:46.104548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118afe60-26cb-4f4e-9091-0b76abbe1453 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.104548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118afe60-26cb-4f4e-9091-0b76abbe1453 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.104548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118afe60-26cb-4f4e-9091-0b76abbe1453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.119715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.120661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.120955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.121239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] PATH_INFO: `/v3/projects/1aac5c827bca4571bc0191a20846041f` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.122513 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aac5c827bca4571bc0191a20846041f)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.124447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.137217 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-118afe60-26cb-4f4e-9091-0b76abbe1453 None None] MFA Rules not processed for user `0e4e0ada75c34457b407a32034ba7aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:46.138479 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-118afe60-26cb-4f4e-9091-0b76abbe1453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:46.188804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aac5c827bca4571bc0191a20846041f', 'request_id': 'req-b437fb37-d0ae-475d-873a-fc676aa51676'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:46.199784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aac5c827bca4571bc0191a20846041f', 'request_id': 'req-b437fb37-d0ae-475d-873a-fc676aa51676'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:46.211891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aac5c827bca4571bc0191a20846041f', 'request_id': 'req-b437fb37-d0ae-475d-873a-fc676aa51676'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:46.234035 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] The token cache is being invalidate because project 1aac5c827bca4571bc0191a20846041f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:46.234493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:46.235739 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:46.242776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1148/2277] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:45 2022] DELETE /identity/v3/projects/1aac5c827bca4571bc0191a20846041f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:46.242776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:46.243844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:46.259998 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:46.260644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1130/2278] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:46.262574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118afe60-26cb-4f4e-9091-0b76abbe1453 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:46.286575 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118afe60-26cb-4f4e-9091-0b76abbe1453 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.337645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437fb37-d0ae-475d-873a-fc676aa51676 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.343915 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4edd94-f27d-42b2-a87a-ce27ca3923bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.345125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4edd94-f27d-42b2-a87a-ce27ca3923bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.345523 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4edd94-f27d-42b2-a87a-ce27ca3923bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.345912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4edd94-f27d-42b2-a87a-ce27ca3923bc admin admin] PATH_INFO: `/v3/projects/5a31b6cb65d34f7cb672e4037de612cb/users/f92a4b9bdc4b4e34bc3fc369b926a2e9/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.366763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4edd94-f27d-42b2-a87a-ce27ca3923bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a31b6cb65d34f7cb672e4037de612cb, user_id=f92a4b9bdc4b4e34bc3fc369b926a2e9, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.370029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4edd94-f27d-42b2-a87a-ce27ca3923bc admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.392885 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1149/2279] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:46 2022] PUT /identity/v3/projects/5a31b6cb65d34f7cb672e4037de612cb/users/f92a4b9bdc4b4e34bc3fc369b926a2e9/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:46.394189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4edd94-f27d-42b2-a87a-ce27ca3923bc admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:46.411056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ec4edd94-f27d-42b2-a87a-ce27ca3923bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:46.449576 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ba08c-9ddd-4aee-a4ec-19e5e2f1fedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.451050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ba08c-9ddd-4aee-a4ec-19e5e2f1fedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.451601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ba08c-9ddd-4aee-a4ec-19e5e2f1fedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.452100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ba08c-9ddd-4aee-a4ec-19e5e2f1fedf admin admin] PATH_INFO: `/v3/users/7851b6ef1a2c480899ed46dac4f4d634` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.454511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ba08c-9ddd-4aee-a4ec-19e5e2f1fedf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7851b6ef1a2c480899ed46dac4f4d634)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.458436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ba08c-9ddd-4aee-a4ec-19e5e2f1fedf admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.493232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4edd94-f27d-42b2-a87a-ce27ca3923bc admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.503691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef63bb-cdfe-4cde-bea4-278ea7c219ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.504458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef63bb-cdfe-4cde-bea4-278ea7c219ab service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.504786 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef63bb-cdfe-4cde-bea4-278ea7c219ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.505119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef63bb-cdfe-4cde-bea4-278ea7c219ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.508586 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1aef63bb-cdfe-4cde-bea4-278ea7c219ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:46.546859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a56ba08c-9ddd-4aee-a4ec-19e5e2f1fedf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7851b6ef1a2c480899ed46dac4f4d634'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:46.557248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a56ba08c-9ddd-4aee-a4ec-19e5e2f1fedf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7851b6ef1a2c480899ed46dac4f4d634'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:46.574042 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:46.590565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a56ba08c-9ddd-4aee-a4ec-19e5e2f1fedf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7851b6ef1a2c480899ed46dac4f4d634'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:46.603744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a56ba08c-9ddd-4aee-a4ec-19e5e2f1fedf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7851b6ef1a2c480899ed46dac4f4d634'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:46.604670 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a56ba08c-9ddd-4aee-a4ec-19e5e2f1fedf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7851b6ef1a2c480899ed46dac4f4d634'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:46.607522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef63bb-cdfe-4cde-bea4-278ea7c219ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.609045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef63bb-cdfe-4cde-bea4-278ea7c219ab service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.614350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1131/2280] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:46 2022] DELETE /identity/v3/users/7851b6ef1a2c480899ed46dac4f4d634 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:46.618795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084caed-5e33-4985-a5c7-aa0507cf4c59 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.619411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084caed-5e33-4985-a5c7-aa0507cf4c59 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.619973 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084caed-5e33-4985-a5c7-aa0507cf4c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.621258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1150/2281] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:46.665673 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7084caed-5e33-4985-a5c7-aa0507cf4c59 None None] MFA Rules not processed for user `f92a4b9bdc4b4e34bc3fc369b926a2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:46.667735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7084caed-5e33-4985-a5c7-aa0507cf4c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:46.686489 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef63bb-cdfe-4cde-bea4-278ea7c219ab service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:46.688281 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1aef63bb-cdfe-4cde-bea4-278ea7c219ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:46.703117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef63bb-cdfe-4cde-bea4-278ea7c219ab service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.763902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:46.784726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1132/2282] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:46.784917 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcb5f7-6284-43de-ae47-d822b07d8911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.785300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcb5f7-6284-43de-ae47-d822b07d8911 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.785542 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcb5f7-6284-43de-ae47-d822b07d8911 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.785925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcb5f7-6284-43de-ae47-d822b07d8911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.789806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-97fcb5f7-6284-43de-ae47-d822b07d8911 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:46.810365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084caed-5e33-4985-a5c7-aa0507cf4c59 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:46.812238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7084caed-5e33-4985-a5c7-aa0507cf4c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:46.832187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084caed-5e33-4985-a5c7-aa0507cf4c59 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.837905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3eec3-8f71-4d3d-809b-e7aa589bd90d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.838692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3eec3-8f71-4d3d-809b-e7aa589bd90d service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.839108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3eec3-8f71-4d3d-809b-e7aa589bd90d service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.839393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3eec3-8f71-4d3d-809b-e7aa589bd90d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.854331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3eec3-8f71-4d3d-809b-e7aa589bd90d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.859326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3eec3-8f71-4d3d-809b-e7aa589bd90d service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.881238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1133/2283] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:46.888899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcb5f7-6284-43de-ae47-d822b07d8911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.890709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcb5f7-6284-43de-ae47-d822b07d8911 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.901887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1151/2284] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:46.919288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3eec3-8f71-4d3d-809b-e7aa589bd90d service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:46.920849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-66e3eec3-8f71-4d3d-809b-e7aa589bd90d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:47.060441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3eec3-8f71-4d3d-809b-e7aa589bd90d service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.075537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a11791-f68e-4e80-9729-fc34ab6198b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.077015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a11791-f68e-4e80-9729-fc34ab6198b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.077600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a11791-f68e-4e80-9729-fc34ab6198b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.078036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a11791-f68e-4e80-9729-fc34ab6198b5 admin admin] PATH_INFO: `/v3/users/f92a4b9bdc4b4e34bc3fc369b926a2e9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.081848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a11791-f68e-4e80-9729-fc34ab6198b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f92a4b9bdc4b4e34bc3fc369b926a2e9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.086832 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a11791-f68e-4e80-9729-fc34ab6198b5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.158782 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcb5f7-6284-43de-ae47-d822b07d8911 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.160628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-97fcb5f7-6284-43de-ae47-d822b07d8911 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:47.173792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a1a11791-f68e-4e80-9729-fc34ab6198b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f92a4b9bdc4b4e34bc3fc369b926a2e9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:47.189642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a1a11791-f68e-4e80-9729-fc34ab6198b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f92a4b9bdc4b4e34bc3fc369b926a2e9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:47.197881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:47.207379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcb5f7-6284-43de-ae47-d822b07d8911 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.224774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ad03e-2305-4402-a34e-2337e077b1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.225678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ad03e-2305-4402-a34e-2337e077b1b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.226056 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ad03e-2305-4402-a34e-2337e077b1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.226459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ad03e-2305-4402-a34e-2337e077b1b8 admin admin] PATH_INFO: `/v3/projects/14b4a2eba9bc497ebea653449476e500` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.228740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ad03e-2305-4402-a34e-2337e077b1b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14b4a2eba9bc497ebea653449476e500)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.231657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ad03e-2305-4402-a34e-2337e077b1b8 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.250969 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a1a11791-f68e-4e80-9729-fc34ab6198b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f92a4b9bdc4b4e34bc3fc369b926a2e9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:47.278795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a1a11791-f68e-4e80-9729-fc34ab6198b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f92a4b9bdc4b4e34bc3fc369b926a2e9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:47.279667 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a1a11791-f68e-4e80-9729-fc34ab6198b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f92a4b9bdc4b4e34bc3fc369b926a2e9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:47.290675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1134/2285] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:46 2022] DELETE /identity/v3/users/f92a4b9bdc4b4e34bc3fc369b926a2e9 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:47.295873 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f2d08-c840-49dd-9dce-b88520c6414a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.296269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f2d08-c840-49dd-9dce-b88520c6414a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.296471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f2d08-c840-49dd-9dce-b88520c6414a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.317570 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-405f2d08-c840-49dd-9dce-b88520c6414a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:47.319197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-405f2d08-c840-49dd-9dce-b88520c6414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:47.350032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d19ad03e-2305-4402-a34e-2337e077b1b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14b4a2eba9bc497ebea653449476e500', 'request_id': 'req-d19ad03e-2305-4402-a34e-2337e077b1b8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:47.374185 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d19ad03e-2305-4402-a34e-2337e077b1b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14b4a2eba9bc497ebea653449476e500', 'request_id': 'req-d19ad03e-2305-4402-a34e-2337e077b1b8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:47.381171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d19ad03e-2305-4402-a34e-2337e077b1b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14b4a2eba9bc497ebea653449476e500', 'request_id': 'req-d19ad03e-2305-4402-a34e-2337e077b1b8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:47.399667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d19ad03e-2305-4402-a34e-2337e077b1b8 admin admin] The token cache is being invalidate because project 14b4a2eba9bc497ebea653449476e500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:47.400606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d19ad03e-2305-4402-a34e-2337e077b1b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:47.402697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d19ad03e-2305-4402-a34e-2337e077b1b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:47.407059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1152/2286] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:47 2022] DELETE /identity/v3/projects/14b4a2eba9bc497ebea653449476e500 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:47.413249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1135/2287] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:47.417165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a02bb-7316-49d0-846c-317f4c3b0a75 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.418111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a02bb-7316-49d0-846c-317f4c3b0a75 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.418209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a02bb-7316-49d0-846c-317f4c3b0a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.428747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f2d08-c840-49dd-9dce-b88520c6414a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.432442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-405f2d08-c840-49dd-9dce-b88520c6414a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:47.450912 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-043a02bb-7316-49d0-846c-317f4c3b0a75 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:47.453008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-043a02bb-7316-49d0-846c-317f4c3b0a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:47.472635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1153/2288] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:47.482142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a5e25-122c-46ef-bb8c-1130cffe6a69 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.486332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a5e25-122c-46ef-bb8c-1130cffe6a69 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.486840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a5e25-122c-46ef-bb8c-1130cffe6a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.529417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ea8a5e25-122c-46ef-bb8c-1130cffe6a69 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:47.535016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ea8a5e25-122c-46ef-bb8c-1130cffe6a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:47.560419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1154/2289] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:47.564621 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a5e25-122c-46ef-bb8c-1130cffe6a69 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.578414 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f2d08-c840-49dd-9dce-b88520c6414a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.582748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a5e25-122c-46ef-bb8c-1130cffe6a69 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.590048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea206dfd-1f8a-4b6f-a05b-ad473f10d2c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.590817 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea206dfd-1f8a-4b6f-a05b-ad473f10d2c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.591289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea206dfd-1f8a-4b6f-a05b-ad473f10d2c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.591664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea206dfd-1f8a-4b6f-a05b-ad473f10d2c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.599542 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726fd89-13be-478d-a280-12dc58334e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.601874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726fd89-13be-478d-a280-12dc58334e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.602624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726fd89-13be-478d-a280-12dc58334e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.603237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726fd89-13be-478d-a280-12dc58334e92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.604262 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726fd89-13be-478d-a280-12dc58334e92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:47.605208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726fd89-13be-478d-a280-12dc58334e92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.607060 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726fd89-13be-478d-a280-12dc58334e92 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.622047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea206dfd-1f8a-4b6f-a05b-ad473f10d2c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.623774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1155/2290] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:47.623972 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea206dfd-1f8a-4b6f-a05b-ad473f10d2c3 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.628269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726fd89-13be-478d-a280-12dc58334e92 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.641412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1136/2291] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:47.642980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726fd89-13be-478d-a280-12dc58334e92 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.666801 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de753fe2-8778-4ca3-ae94-76f9816370fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.667476 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de753fe2-8778-4ca3-ae94-76f9816370fb admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.667760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de753fe2-8778-4ca3-ae94-76f9816370fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.668041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de753fe2-8778-4ca3-ae94-76f9816370fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.676566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de753fe2-8778-4ca3-ae94-76f9816370fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.676566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de753fe2-8778-4ca3-ae94-76f9816370fb admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.723293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1156/2292] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:47 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:47.725685 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea206dfd-1f8a-4b6f-a05b-ad473f10d2c3 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.747589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea206dfd-1f8a-4b6f-a05b-ad473f10d2c3 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.828902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599010d-a1f3-44aa-b18c-386b9a485e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.829566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599010d-a1f3-44aa-b18c-386b9a485e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.829853 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599010d-a1f3-44aa-b18c-386b9a485e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.830134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599010d-a1f3-44aa-b18c-386b9a485e19 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.830830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599010d-a1f3-44aa-b18c-386b9a485e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.832901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599010d-a1f3-44aa-b18c-386b9a485e19 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.860865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1137/2293] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:47 2022] POST /identity/v3/users => generated 284 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:47.864164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de753fe2-8778-4ca3-ae94-76f9816370fb admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.878406 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de753fe2-8778-4ca3-ae94-76f9816370fb admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.884267 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a074f3-c77c-419e-870e-fcb710312e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.885089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a074f3-c77c-419e-870e-fcb710312e1b admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.885449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a074f3-c77c-419e-870e-fcb710312e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.885802 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a074f3-c77c-419e-870e-fcb710312e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.886625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a074f3-c77c-419e-870e-fcb710312e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.888381 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a074f3-c77c-419e-870e-fcb710312e1b admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.899952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1157/2294] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:47.903997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599010d-a1f3-44aa-b18c-386b9a485e19 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.914926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599010d-a1f3-44aa-b18c-386b9a485e19 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.919574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5445d32-2ea1-46c8-8dfb-625285e28871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.920249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5445d32-2ea1-46c8-8dfb-625285e28871 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.920565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5445d32-2ea1-46c8-8dfb-625285e28871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.920846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5445d32-2ea1-46c8-8dfb-625285e28871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.921587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5445d32-2ea1-46c8-8dfb-625285e28871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.923196 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5445d32-2ea1-46c8-8dfb-625285e28871 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.926219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:47.941156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1138/2295] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:47.942365 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a074f3-c77c-419e-870e-fcb710312e1b admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.954281 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a074f3-c77c-419e-870e-fcb710312e1b admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.959542 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad77af-72a8-4316-a9ec-e5984703cedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.961019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad77af-72a8-4316-a9ec-e5984703cedf admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.961019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad77af-72a8-4316-a9ec-e5984703cedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.961019 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad77af-72a8-4316-a9ec-e5984703cedf admin admin] PATH_INFO: `/v3/projects/5f9272ab221d45aead1718872b27e545/users/454c63976a6e4dd981e93d6912115c42/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.979549 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad77af-72a8-4316-a9ec-e5984703cedf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9272ab221d45aead1718872b27e545, user_id=454c63976a6e4dd981e93d6912115c42, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.982341 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad77af-72a8-4316-a9ec-e5984703cedf admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.984693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:47.994436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5445d32-2ea1-46c8-8dfb-625285e28871 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.996833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d5445d32-2ea1-46c8-8dfb-625285e28871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:48.012528 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1158/2296] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:47 2022] PUT /identity/v3/projects/5f9272ab221d45aead1718872b27e545/users/454c63976a6e4dd981e93d6912115c42/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:48.012846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5445d32-2ea1-46c8-8dfb-625285e28871 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.021097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b272f-b743-4561-8cc3-a02fd17055b7 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.021392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b272f-b743-4561-8cc3-a02fd17055b7 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.021679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b272f-b743-4561-8cc3-a02fd17055b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.050586 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-938b272f-b743-4561-8cc3-a02fd17055b7 None None] MFA Rules not processed for user `454c63976a6e4dd981e93d6912115c42`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:48.051805 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-938b272f-b743-4561-8cc3-a02fd17055b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:48.082557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c7f95-0aac-4d0b-ad12-84c23e2a8191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.083440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c7f95-0aac-4d0b-ad12-84c23e2a8191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.083731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c7f95-0aac-4d0b-ad12-84c23e2a8191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.084012 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c7f95-0aac-4d0b-ad12-84c23e2a8191 admin admin] PATH_INFO: `/v3/projects/5a31b6cb65d34f7cb672e4037de612cb` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.085244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c7f95-0aac-4d0b-ad12-84c23e2a8191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a31b6cb65d34f7cb672e4037de612cb)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.087231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c7f95-0aac-4d0b-ad12-84c23e2a8191 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.122910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1159/2297] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:48.136407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b272f-b743-4561-8cc3-a02fd17055b7 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.137304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-751c7f95-0aac-4d0b-ad12-84c23e2a8191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a31b6cb65d34f7cb672e4037de612cb', 'request_id': 'req-751c7f95-0aac-4d0b-ad12-84c23e2a8191'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:48.137794 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-938b272f-b743-4561-8cc3-a02fd17055b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:48.142981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-751c7f95-0aac-4d0b-ad12-84c23e2a8191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a31b6cb65d34f7cb672e4037de612cb', 'request_id': 'req-751c7f95-0aac-4d0b-ad12-84c23e2a8191'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:48.150233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-751c7f95-0aac-4d0b-ad12-84c23e2a8191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a31b6cb65d34f7cb672e4037de612cb', 'request_id': 'req-751c7f95-0aac-4d0b-ad12-84c23e2a8191'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:48.161678 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-751c7f95-0aac-4d0b-ad12-84c23e2a8191 admin admin] The token cache is being invalidate because project 5a31b6cb65d34f7cb672e4037de612cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:48.161965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-751c7f95-0aac-4d0b-ad12-84c23e2a8191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:48.162622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-751c7f95-0aac-4d0b-ad12-84c23e2a8191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:48.164543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1139/2298] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:47 2022] DELETE /identity/v3/projects/5a31b6cb65d34f7cb672e4037de612cb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:48.169147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27466efc-4875-4082-aa83-a08cf95d202b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.169439 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27466efc-4875-4082-aa83-a08cf95d202b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.169719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27466efc-4875-4082-aa83-a08cf95d202b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.188849 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-27466efc-4875-4082-aa83-a08cf95d202b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:48.190141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-27466efc-4875-4082-aa83-a08cf95d202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:48.199068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b272f-b743-4561-8cc3-a02fd17055b7 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.257421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1140/2299] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:48.264546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc6d6c-3f00-400e-84cc-a3d602765712 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b67db080874f3ebb508239c7a8ccf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.265848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc6d6c-3f00-400e-84cc-a3d602765712 service glance] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.266390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc6d6c-3f00-400e-84cc-a3d602765712 service glance] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.266887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc6d6c-3f00-400e-84cc-a3d602765712 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.266981 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8123c7-cb0b-469a-b3cc-333f2f2882a6 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.267178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8123c7-cb0b-469a-b3cc-333f2f2882a6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.267454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8123c7-cb0b-469a-b3cc-333f2f2882a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.270175 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8cbc6d6c-3f00-400e-84cc-a3d602765712 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:48.288313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8a8123c7-cb0b-469a-b3cc-333f2f2882a6 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:48.290109 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8a8123c7-cb0b-469a-b3cc-333f2f2882a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:48.298766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1141/2300] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:48.303088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8123c7-cb0b-469a-b3cc-333f2f2882a6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.314209 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8123c7-cb0b-469a-b3cc-333f2f2882a6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.322589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25d332-2b98-462d-b7df-b8b9ffcacf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.323440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25d332-2b98-462d-b7df-b8b9ffcacf6f admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.323872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25d332-2b98-462d-b7df-b8b9ffcacf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.324374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25d332-2b98-462d-b7df-b8b9ffcacf6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.325695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b25d332-2b98-462d-b7df-b8b9ffcacf6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:48.326386 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b25d332-2b98-462d-b7df-b8b9ffcacf6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.328871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b25d332-2b98-462d-b7df-b8b9ffcacf6f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.348079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1142/2301] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:48.354341 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25d332-2b98-462d-b7df-b8b9ffcacf6f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.363139 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:48.369536 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc6d6c-3f00-400e-84cc-a3d602765712 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.370186 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25d332-2b98-462d-b7df-b8b9ffcacf6f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.372341 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc6d6c-3f00-400e-84cc-a3d602765712 service glance] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.375624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94f44d-3324-4371-9bfc-3214c474f91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.376572 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94f44d-3324-4371-9bfc-3214c474f91f admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.377033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94f44d-3324-4371-9bfc-3214c474f91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.377491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94f44d-3324-4371-9bfc-3214c474f91f admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.378702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94f44d-3324-4371-9bfc-3214c474f91f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.383398 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94f44d-3324-4371-9bfc-3214c474f91f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.390639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1160/2302] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:48.397854 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:48.450675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1143/2303] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:48 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:48.455883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc6d6c-3f00-400e-84cc-a3d602765712 service glance] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.476428 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc6d6c-3f00-400e-84cc-a3d602765712 service glance] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.576668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32359a61-cbd3-4c37-8625-5efde9667a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.578245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32359a61-cbd3-4c37-8625-5efde9667a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.578953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32359a61-cbd3-4c37-8625-5efde9667a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.579560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32359a61-cbd3-4c37-8625-5efde9667a58 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.580678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32359a61-cbd3-4c37-8625-5efde9667a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.582677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32359a61-cbd3-4c37-8625-5efde9667a58 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.620951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1161/2304] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:48 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:48.626693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94f44d-3324-4371-9bfc-3214c474f91f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.649434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94f44d-3324-4371-9bfc-3214c474f91f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.659991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0262f-0cbc-4b96-b597-a620545077ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.661687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0262f-0cbc-4b96-b597-a620545077ef admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.661863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0262f-0cbc-4b96-b597-a620545077ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.662379 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0262f-0cbc-4b96-b597-a620545077ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.665335 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0262f-0cbc-4b96-b597-a620545077ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.668585 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0262f-0cbc-4b96-b597-a620545077ef admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.687690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1144/2305] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:48.691642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32359a61-cbd3-4c37-8625-5efde9667a58 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.717083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32359a61-cbd3-4c37-8625-5efde9667a58 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.724336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb49d3-4531-4b13-8e15-c249274c678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.728011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb49d3-4531-4b13-8e15-c249274c678f admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.728515 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb49d3-4531-4b13-8e15-c249274c678f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.728931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb49d3-4531-4b13-8e15-c249274c678f admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.729913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb49d3-4531-4b13-8e15-c249274c678f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.731985 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb49d3-4531-4b13-8e15-c249274c678f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.747088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1162/2306] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:48.748352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0262f-0cbc-4b96-b597-a620545077ef admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.767507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0262f-0cbc-4b96-b597-a620545077ef admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.780347 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d1ea0-eb53-455d-a3b1-996e46cdb478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.781505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d1ea0-eb53-455d-a3b1-996e46cdb478 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.781891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d1ea0-eb53-455d-a3b1-996e46cdb478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.782348 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d1ea0-eb53-455d-a3b1-996e46cdb478 admin admin] PATH_INFO: `/v3/projects/8da551e8b23d443884cd85c4b0f9a1dd/users/1f9dae20f82448a4b09189f4c55ef481/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.818476 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d1ea0-eb53-455d-a3b1-996e46cdb478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da551e8b23d443884cd85c4b0f9a1dd, user_id=1f9dae20f82448a4b09189f4c55ef481, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.827532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d1ea0-eb53-455d-a3b1-996e46cdb478 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.862976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1145/2307] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:48 2022] PUT /identity/v3/projects/8da551e8b23d443884cd85c4b0f9a1dd/users/1f9dae20f82448a4b09189f4c55ef481/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:48.880724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e50a56-9d24-45cc-b6b3-2203f1c25d3b None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.881167 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e50a56-9d24-45cc-b6b3-2203f1c25d3b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.881322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e50a56-9d24-45cc-b6b3-2203f1c25d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.936856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-81e50a56-9d24-45cc-b6b3-2203f1c25d3b None None] MFA Rules not processed for user `1f9dae20f82448a4b09189f4c55ef481`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:48.938667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-81e50a56-9d24-45cc-b6b3-2203f1c25d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:49.060631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1163/2308] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:49.065274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d1ea0-eb53-455d-a3b1-996e46cdb478 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.222806 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d1ea0-eb53-455d-a3b1-996e46cdb478 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.228997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac1740-501b-4d43-92b9-0a5ee1159e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.229785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac1740-501b-4d43-92b9-0a5ee1159e3d admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.230146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac1740-501b-4d43-92b9-0a5ee1159e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.230498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac1740-501b-4d43-92b9-0a5ee1159e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.231307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac1740-501b-4d43-92b9-0a5ee1159e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.233688 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac1740-501b-4d43-92b9-0a5ee1159e3d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.295905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1146/2309] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:49 2022] POST /identity/v3/projects => generated 375 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:49.302714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e50a56-9d24-45cc-b6b3-2203f1c25d3b None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.327277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e50a56-9d24-45cc-b6b3-2203f1c25d3b None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.429704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37839b-6dd3-4aff-96d8-54add52dcd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.431434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37839b-6dd3-4aff-96d8-54add52dcd47 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.432280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37839b-6dd3-4aff-96d8-54add52dcd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.433087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37839b-6dd3-4aff-96d8-54add52dcd47 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.434766 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37839b-6dd3-4aff-96d8-54add52dcd47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.439327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37839b-6dd3-4aff-96d8-54add52dcd47 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.454500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:49.477759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1164/2310] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:49 2022] POST /identity/v3/users => generated 294 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:49.481779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac1740-501b-4d43-92b9-0a5ee1159e3d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.494494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac1740-501b-4d43-92b9-0a5ee1159e3d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.499721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18a78c-df3f-4923-b24f-245800a6a85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.500417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18a78c-df3f-4923-b24f-245800a6a85e admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.500749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18a78c-df3f-4923-b24f-245800a6a85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.501030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18a78c-df3f-4923-b24f-245800a6a85e admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.501791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18a78c-df3f-4923-b24f-245800a6a85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.503470 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18a78c-df3f-4923-b24f-245800a6a85e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.516686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1147/2311] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:49.521220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37839b-6dd3-4aff-96d8-54add52dcd47 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.533101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37839b-6dd3-4aff-96d8-54add52dcd47 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.538097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f79bd8-57dd-4b25-b33e-ec31d7c61dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.539297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f79bd8-57dd-4b25-b33e-ec31d7c61dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.539760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f79bd8-57dd-4b25-b33e-ec31d7c61dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.540047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f79bd8-57dd-4b25-b33e-ec31d7c61dc3 admin admin] PATH_INFO: `/v3/projects/39b6f4b1689c41349866eb0d627b386a/users/6a755398f54543be89f2a003fe163c5d/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.557703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f79bd8-57dd-4b25-b33e-ec31d7c61dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b6f4b1689c41349866eb0d627b386a, user_id=6a755398f54543be89f2a003fe163c5d, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.560506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f79bd8-57dd-4b25-b33e-ec31d7c61dc3 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.578359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1165/2312] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:49 2022] PUT /identity/v3/projects/39b6f4b1689c41349866eb0d627b386a/users/6a755398f54543be89f2a003fe163c5d/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:49.581530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18a78c-df3f-4923-b24f-245800a6a85e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.594003 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18a78c-df3f-4923-b24f-245800a6a85e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.661552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3f66f-04cb-44c0-ab1c-77175cc2c3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.662242 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3f66f-04cb-44c0-ab1c-77175cc2c3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.662532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3f66f-04cb-44c0-ab1c-77175cc2c3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.662814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3f66f-04cb-44c0-ab1c-77175cc2c3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.663627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3f66f-04cb-44c0-ab1c-77175cc2c3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.665922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3f66f-04cb-44c0-ab1c-77175cc2c3c1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.682558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1148/2313] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:49.683897 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f79bd8-57dd-4b25-b33e-ec31d7c61dc3 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.696328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f79bd8-57dd-4b25-b33e-ec31d7c61dc3 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.701954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e66015-f9c9-4af9-98fb-0c18b327ce74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.703011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e66015-f9c9-4af9-98fb-0c18b327ce74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.703180 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e66015-f9c9-4af9-98fb-0c18b327ce74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.703423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e66015-f9c9-4af9-98fb-0c18b327ce74 admin admin] PATH_INFO: `/v3/projects/39b6f4b1689c41349866eb0d627b386a/users/6a755398f54543be89f2a003fe163c5d/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.705743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e66015-f9c9-4af9-98fb-0c18b327ce74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b6f4b1689c41349866eb0d627b386a, user_id=6a755398f54543be89f2a003fe163c5d, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.708572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e66015-f9c9-4af9-98fb-0c18b327ce74 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.736285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1166/2314] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:49 2022] PUT /identity/v3/projects/39b6f4b1689c41349866eb0d627b386a/users/6a755398f54543be89f2a003fe163c5d/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:49.750717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20845d5d-ef23-4383-b02d-587374308e9c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.751125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20845d5d-ef23-4383-b02d-587374308e9c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.751508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20845d5d-ef23-4383-b02d-587374308e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.783831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-20845d5d-ef23-4383-b02d-587374308e9c None None] MFA Rules not processed for user `6a755398f54543be89f2a003fe163c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:49.785466 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-20845d5d-ef23-4383-b02d-587374308e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:49.798283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:49.881755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1149/2315] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:49.895570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e66015-f9c9-4af9-98fb-0c18b327ce74 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.899201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b4e66015-f9c9-4af9-98fb-0c18b327ce74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:49.987436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e66015-f9c9-4af9-98fb-0c18b327ce74 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.998028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed24e3a-23c5-4d85-85e4-f2378dba2708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.998880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed24e3a-23c5-4d85-85e4-f2378dba2708 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.999166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed24e3a-23c5-4d85-85e4-f2378dba2708 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.999448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed24e3a-23c5-4d85-85e4-f2378dba2708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.073106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed24e3a-23c5-4d85-85e4-f2378dba2708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:50.075033 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed24e3a-23c5-4d85-85e4-f2378dba2708 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:50.086836 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1167/2316] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:50.559104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20845d5d-ef23-4383-b02d-587374308e9c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:50.560587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-20845d5d-ef23-4383-b02d-587374308e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:50.636380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20845d5d-ef23-4383-b02d-587374308e9c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:50.649902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:50.650820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.651123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.651396 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] PATH_INFO: `/v3/users/e3c3383707e34f3c9d3707b52143202b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.652744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c3383707e34f3c9d3707b52143202b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:50.654717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:50.709560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c3383707e34f3c9d3707b52143202b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:50.716976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c3383707e34f3c9d3707b52143202b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:50.732442 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c3383707e34f3c9d3707b52143202b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:50.741313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c3383707e34f3c9d3707b52143202b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:50.743445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c3383707e34f3c9d3707b52143202b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:50.751048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1150/2317] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:50 2022] DELETE /identity/v3/users/e3c3383707e34f3c9d3707b52143202b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:50.756074 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a074-d468-418f-b955-14c7de1be672 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.756853 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a074-d468-418f-b955-14c7de1be672 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.757069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a074-d468-418f-b955-14c7de1be672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.775980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-9dc8a074-d468-418f-b955-14c7de1be672 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:50.778002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9dc8a074-d468-418f-b955-14c7de1be672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:50.849869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1168/2318] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:50.862262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:50.863938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:50.899902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:50.934643 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c597a0-9cd1-477e-b3b8-ed74fa9fa566 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:50.938336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc0b23-29b1-4dde-8288-1aa44ea9a22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:50.938993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc0b23-29b1-4dde-8288-1aa44ea9a22f service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.939311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc0b23-29b1-4dde-8288-1aa44ea9a22f service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.939588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc0b23-29b1-4dde-8288-1aa44ea9a22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.950175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc0b23-29b1-4dde-8288-1aa44ea9a22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:50.951735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc0b23-29b1-4dde-8288-1aa44ea9a22f service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:50.963104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1151/2319] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:51.500325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8a074-d468-418f-b955-14c7de1be672 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.504936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9dc8a074-d468-418f-b955-14c7de1be672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:51.520311 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8a074-d468-418f-b955-14c7de1be672 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.529839 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.530843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.531203 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.531571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] PATH_INFO: `/v3/projects/5360faa4319b445cbfbb3bc3581e0e1d` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.534752 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5360faa4319b445cbfbb3bc3581e0e1d)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.537882 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.541977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:51.692831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5360faa4319b445cbfbb3bc3581e0e1d', 'request_id': 'req-2ea724be-b715-47c4-8050-d785c7871857'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:51.710077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5360faa4319b445cbfbb3bc3581e0e1d', 'request_id': 'req-2ea724be-b715-47c4-8050-d785c7871857'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:51.724290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5360faa4319b445cbfbb3bc3581e0e1d', 'request_id': 'req-2ea724be-b715-47c4-8050-d785c7871857'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:51.746362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] The token cache is being invalidate because project 5360faa4319b445cbfbb3bc3581e0e1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:51.746362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:51.748831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:51.752224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1169/2320] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:51 2022] DELETE /identity/v3/projects/5360faa4319b445cbfbb3bc3581e0e1d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:51.756603 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc0b23-29b1-4dde-8288-1aa44ea9a22f service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.767692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc0b23-29b1-4dde-8288-1aa44ea9a22f service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.850547 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.851716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.852175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.852911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] PATH_INFO: `/v3/users/f9d512bc874949ceaded0682d0f68e90` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.854835 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9d512bc874949ceaded0682d0f68e90)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.857423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.924144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9d512bc874949ceaded0682d0f68e90'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:51.932513 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9d512bc874949ceaded0682d0f68e90'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:51.950315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9d512bc874949ceaded0682d0f68e90'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:51.960179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9d512bc874949ceaded0682d0f68e90'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:51.961057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9d512bc874949ceaded0682d0f68e90'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:51.967977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1152/2321] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:51 2022] DELETE /identity/v3/users/f9d512bc874949ceaded0682d0f68e90 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:52.250910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:52.252796 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:52.268108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea724be-b715-47c4-8050-d785c7871857 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:52.341464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:52.342727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:52.343055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:52.343384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] PATH_INFO: `/v3/projects/7d28f1b462824d96ad689a5804d08771` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:52.345992 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d28f1b462824d96ad689a5804d08771)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:52.346800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:52.409980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d28f1b462824d96ad689a5804d08771', 'request_id': 'req-26755c3c-ab01-4306-8cdd-e32178ecd45e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:52.430172 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d28f1b462824d96ad689a5804d08771', 'request_id': 'req-26755c3c-ab01-4306-8cdd-e32178ecd45e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:52.440413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d28f1b462824d96ad689a5804d08771', 'request_id': 'req-26755c3c-ab01-4306-8cdd-e32178ecd45e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:52.468190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] The token cache is being invalidate because project 7d28f1b462824d96ad689a5804d08771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:52.468467 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:52.469273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:52.471441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1170/2322] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:52 2022] DELETE /identity/v3/projects/7d28f1b462824d96ad689a5804d08771 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:52.476842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:52.478736 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:52.492459 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d885e-42b8-4704-b6f3-60111a2519ef admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:52.546790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:52.563860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:52.564893 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:52.565421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:52.565694 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] PATH_INFO: `/v3/users/0e4e0ada75c34457b407a32034ba7aa8` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:52.567174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e4e0ada75c34457b407a32034ba7aa8)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:52.569199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:52.630143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e4e0ada75c34457b407a32034ba7aa8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:52.638072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e4e0ada75c34457b407a32034ba7aa8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:52.662074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e4e0ada75c34457b407a32034ba7aa8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:52.682120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e4e0ada75c34457b407a32034ba7aa8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:52.682856 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e4e0ada75c34457b407a32034ba7aa8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:52.691291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1153/2323] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:52 2022] DELETE /identity/v3/users/0e4e0ada75c34457b407a32034ba7aa8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:52.915017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:52.916617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:52.931217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26755c3c-ab01-4306-8cdd-e32178ecd45e admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:52.966506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:53.022208 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da06e4-d05d-4a67-82a6-0a875511701f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.024426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da06e4-d05d-4a67-82a6-0a875511701f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.024890 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da06e4-d05d-4a67-82a6-0a875511701f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.025142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da06e4-d05d-4a67-82a6-0a875511701f admin admin] PATH_INFO: `/v3/users/1f9dae20f82448a4b09189f4c55ef481` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.026973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da06e4-d05d-4a67-82a6-0a875511701f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f9dae20f82448a4b09189f4c55ef481)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.028744 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da06e4-d05d-4a67-82a6-0a875511701f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.108532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-54da06e4-d05d-4a67-82a6-0a875511701f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f9dae20f82448a4b09189f4c55ef481'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:53.119442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-54da06e4-d05d-4a67-82a6-0a875511701f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f9dae20f82448a4b09189f4c55ef481'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:53.120721 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-54da06e4-d05d-4a67-82a6-0a875511701f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f9dae20f82448a4b09189f4c55ef481'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:53.131896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-54da06e4-d05d-4a67-82a6-0a875511701f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f9dae20f82448a4b09189f4c55ef481'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:53.132728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-54da06e4-d05d-4a67-82a6-0a875511701f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f9dae20f82448a4b09189f4c55ef481'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:53.148373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.150393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:53.165263 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1171/2324] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:52 2022] DELETE /identity/v3/users/1f9dae20f82448a4b09189f4c55ef481 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:53.175884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65839cd2-8880-42d0-ba42-cae8f7aa7d0c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.183546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf12b4c-2677-499a-a91b-7874bcf923e3 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.183743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf12b4c-2677-499a-a91b-7874bcf923e3 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.184054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf12b4c-2677-499a-a91b-7874bcf923e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.218595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-faf12b4c-2677-499a-a91b-7874bcf923e3 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:53.219823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-faf12b4c-2677-499a-a91b-7874bcf923e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:53.260291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e1904-6863-4910-8dc6-0e854f1bd57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.261668 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e1904-6863-4910-8dc6-0e854f1bd57b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.262178 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e1904-6863-4910-8dc6-0e854f1bd57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.262537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e1904-6863-4910-8dc6-0e854f1bd57b admin admin] PATH_INFO: `/v3/projects/b5bac759b6de4bac8faaf7afc3a2621b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.264224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e1904-6863-4910-8dc6-0e854f1bd57b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5bac759b6de4bac8faaf7afc3a2621b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.266468 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e1904-6863-4910-8dc6-0e854f1bd57b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.299390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1172/2325] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:53.311210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf12b4c-2677-499a-a91b-7874bcf923e3 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.314156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-faf12b4c-2677-499a-a91b-7874bcf923e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:53.343499 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1c6e1904-6863-4910-8dc6-0e854f1bd57b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5bac759b6de4bac8faaf7afc3a2621b', 'request_id': 'req-1c6e1904-6863-4910-8dc6-0e854f1bd57b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:53.354489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1c6e1904-6863-4910-8dc6-0e854f1bd57b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5bac759b6de4bac8faaf7afc3a2621b', 'request_id': 'req-1c6e1904-6863-4910-8dc6-0e854f1bd57b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:53.366923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1c6e1904-6863-4910-8dc6-0e854f1bd57b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5bac759b6de4bac8faaf7afc3a2621b', 'request_id': 'req-1c6e1904-6863-4910-8dc6-0e854f1bd57b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:53.373406 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:53.380938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:53.390194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1c6e1904-6863-4910-8dc6-0e854f1bd57b admin admin] The token cache is being invalidate because project b5bac759b6de4bac8faaf7afc3a2621b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:53.390942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1c6e1904-6863-4910-8dc6-0e854f1bd57b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:53.391640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1c6e1904-6863-4910-8dc6-0e854f1bd57b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:53.392272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf12b4c-2677-499a-a91b-7874bcf923e3 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.393278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1154/2326] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:53 2022] DELETE /identity/v3/projects/b5bac759b6de4bac8faaf7afc3a2621b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:53.400308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3b72a-2ed7-44e0-b3d9-0b1442bcd15e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.400640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3b72a-2ed7-44e0-b3d9-0b1442bcd15e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.400923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3b72a-2ed7-44e0-b3d9-0b1442bcd15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.421876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-bbd3b72a-2ed7-44e0-b3d9-0b1442bcd15e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:53.423339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bbd3b72a-2ed7-44e0-b3d9-0b1442bcd15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:53.462448 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87853955-1ee8-457e-95ac-6841cf7fe005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.463168 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87853955-1ee8-457e-95ac-6841cf7fe005 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.463458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87853955-1ee8-457e-95ac-6841cf7fe005 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.463739 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87853955-1ee8-457e-95ac-6841cf7fe005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.465751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-87853955-1ee8-457e-95ac-6841cf7fe005 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:53.517115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1155/2327] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:53.525751 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50c801-5f56-49f0-829b-b757239f876b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.526202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50c801-5f56-49f0-829b-b757239f876b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.526584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50c801-5f56-49f0-829b-b757239f876b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.553711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87853955-1ee8-457e-95ac-6841cf7fe005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.555862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87853955-1ee8-457e-95ac-6841cf7fe005 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.557544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ff50c801-5f56-49f0-829b-b757239f876b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:53.559705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ff50c801-5f56-49f0-829b-b757239f876b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:53.568466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1173/2328] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:53.575431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1156/2329] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:53.584292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87853955-1ee8-457e-95ac-6841cf7fe005 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.603168 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87853955-1ee8-457e-95ac-6841cf7fe005 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.610390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b365e0a-707e-414a-adab-224a4ea42c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.611554 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b365e0a-707e-414a-adab-224a4ea42c1c admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.613371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b365e0a-707e-414a-adab-224a4ea42c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.613371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b365e0a-707e-414a-adab-224a4ea42c1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.613583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b365e0a-707e-414a-adab-224a4ea42c1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:53.614108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b365e0a-707e-414a-adab-224a4ea42c1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.617394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b365e0a-707e-414a-adab-224a4ea42c1c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.642106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1174/2330] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:53.647383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50c801-5f56-49f0-829b-b757239f876b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.659491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50c801-5f56-49f0-829b-b757239f876b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.667144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79712c79-237e-4dae-85b3-37847d92814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.667535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712c79-237e-4dae-85b3-37847d92814d admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.667827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712c79-237e-4dae-85b3-37847d92814d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.668717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712c79-237e-4dae-85b3-37847d92814d admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.671763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79712c79-237e-4dae-85b3-37847d92814d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.672728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79712c79-237e-4dae-85b3-37847d92814d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.744208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1157/2331] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:53 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:53.750841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b365e0a-707e-414a-adab-224a4ea42c1c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.771265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b365e0a-707e-414a-adab-224a4ea42c1c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.898242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007e7db-11d2-445b-a026-64fa22ba59c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.898950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007e7db-11d2-445b-a026-64fa22ba59c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.899315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007e7db-11d2-445b-a026-64fa22ba59c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.899712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007e7db-11d2-445b-a026-64fa22ba59c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.900441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007e7db-11d2-445b-a026-64fa22ba59c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.902431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007e7db-11d2-445b-a026-64fa22ba59c4 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.930999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1175/2332] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:53 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:53.935132 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79712c79-237e-4dae-85b3-37847d92814d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.947293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79712c79-237e-4dae-85b3-37847d92814d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.953085 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624fd8f-28d0-49b8-bf14-d081e77b0457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.953776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624fd8f-28d0-49b8-bf14-d081e77b0457 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.954059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624fd8f-28d0-49b8-bf14-d081e77b0457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.954335 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624fd8f-28d0-49b8-bf14-d081e77b0457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.955075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624fd8f-28d0-49b8-bf14-d081e77b0457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.956973 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624fd8f-28d0-49b8-bf14-d081e77b0457 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.970640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1158/2333] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:53.977846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007e7db-11d2-445b-a026-64fa22ba59c4 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.989270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007e7db-11d2-445b-a026-64fa22ba59c4 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.995462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f970b-e2ba-4a68-bb74-871aae666e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.996346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f970b-e2ba-4a68-bb74-871aae666e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.996733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f970b-e2ba-4a68-bb74-871aae666e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.997130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f970b-e2ba-4a68-bb74-871aae666e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.997927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f970b-e2ba-4a68-bb74-871aae666e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.999573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f970b-e2ba-4a68-bb74-871aae666e10 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.014159 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1176/2334] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:54.019691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624fd8f-28d0-49b8-bf14-d081e77b0457 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.051869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624fd8f-28d0-49b8-bf14-d081e77b0457 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.061623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac19d4f-5e0a-4dd9-8e29-faa3164a2f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.063211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac19d4f-5e0a-4dd9-8e29-faa3164a2f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.063717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac19d4f-5e0a-4dd9-8e29-faa3164a2f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.064223 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac19d4f-5e0a-4dd9-8e29-faa3164a2f27 admin admin] PATH_INFO: `/v3/projects/feefb78f565b435fb4731d897ba0ddef/users/03cdb5e029f64dcf976b8dc2eb7fc8f2/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.093771 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac19d4f-5e0a-4dd9-8e29-faa3164a2f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feefb78f565b435fb4731d897ba0ddef, user_id=03cdb5e029f64dcf976b8dc2eb7fc8f2, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.099635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac19d4f-5e0a-4dd9-8e29-faa3164a2f27 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.121222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1159/2335] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:54 2022] PUT /identity/v3/projects/feefb78f565b435fb4731d897ba0ddef/users/03cdb5e029f64dcf976b8dc2eb7fc8f2/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:54.134608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df21101-9569-4592-9b59-e15ab8489fc2 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.135290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df21101-9569-4592-9b59-e15ab8489fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.135843 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df21101-9569-4592-9b59-e15ab8489fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.166491 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:54.175292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-6df21101-9569-4592-9b59-e15ab8489fc2 None None] MFA Rules not processed for user `03cdb5e029f64dcf976b8dc2eb7fc8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:54.177342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6df21101-9569-4592-9b59-e15ab8489fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:54.279905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1177/2336] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:54.284526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac19d4f-5e0a-4dd9-8e29-faa3164a2f27 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.370949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac19d4f-5e0a-4dd9-8e29-faa3164a2f27 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.376499 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a14836-4a03-419b-84ab-c7228a5d5d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.377539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a14836-4a03-419b-84ab-c7228a5d5d88 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.378057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a14836-4a03-419b-84ab-c7228a5d5d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.378482 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a14836-4a03-419b-84ab-c7228a5d5d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.379358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a14836-4a03-419b-84ab-c7228a5d5d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.382208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a14836-4a03-419b-84ab-c7228a5d5d88 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.425964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1160/2337] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:54 2022] POST /identity/v3/projects => generated 379 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:54.430991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df21101-9569-4592-9b59-e15ab8489fc2 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.442276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df21101-9569-4592-9b59-e15ab8489fc2 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.453280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a14836-4a03-419b-84ab-c7228a5d5d88 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.454995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-30a14836-4a03-419b-84ab-c7228a5d5d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:54.487225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:54.528740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d624c51-dde1-4bdf-8f28-dfab41195c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.529432 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d624c51-dde1-4bdf-8f28-dfab41195c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.529717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d624c51-dde1-4bdf-8f28-dfab41195c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.529995 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d624c51-dde1-4bdf-8f28-dfab41195c46 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.530737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d624c51-dde1-4bdf-8f28-dfab41195c46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.532678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d624c51-dde1-4bdf-8f28-dfab41195c46 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.537931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a14836-4a03-419b-84ab-c7228a5d5d88 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.544150 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.545088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.545418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.545725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] PATH_INFO: `/v3/projects/8da551e8b23d443884cd85c4b0f9a1dd` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.546952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8da551e8b23d443884cd85c4b0f9a1dd)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.549018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.558014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1178/2338] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:54 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:54.559334 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d624c51-dde1-4bdf-8f28-dfab41195c46 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.570316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d624c51-dde1-4bdf-8f28-dfab41195c46 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.575434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c8062-fc23-4801-bbf8-e3448fd54bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.576059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c8062-fc23-4801-bbf8-e3448fd54bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.576348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c8062-fc23-4801-bbf8-e3448fd54bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.576661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c8062-fc23-4801-bbf8-e3448fd54bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.577411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c8062-fc23-4801-bbf8-e3448fd54bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.579012 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c8062-fc23-4801-bbf8-e3448fd54bb0 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.590303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1179/2339] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:54.594475 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c8062-fc23-4801-bbf8-e3448fd54bb0 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.604823 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8da551e8b23d443884cd85c4b0f9a1dd', 'request_id': 'req-c497cc11-2b1b-450a-a61a-45ec5679402e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:54.605899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c8062-fc23-4801-bbf8-e3448fd54bb0 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.612875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8da551e8b23d443884cd85c4b0f9a1dd', 'request_id': 'req-c497cc11-2b1b-450a-a61a-45ec5679402e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:54.628687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8da551e8b23d443884cd85c4b0f9a1dd', 'request_id': 'req-c497cc11-2b1b-450a-a61a-45ec5679402e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:54.643198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] The token cache is being invalidate because project 8da551e8b23d443884cd85c4b0f9a1dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:54.643829 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:54.644884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:54.646721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1161/2340] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:54 2022] DELETE /identity/v3/projects/8da551e8b23d443884cd85c4b0f9a1dd => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:54.650977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.652350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:54.678502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790b04f-da65-4d69-a67b-fded7b8493aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.680235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790b04f-da65-4d69-a67b-fded7b8493aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.680235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790b04f-da65-4d69-a67b-fded7b8493aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.680522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790b04f-da65-4d69-a67b-fded7b8493aa admin admin] PATH_INFO: `/v3/projects/133912be27ab446b9194a7d75dc6475f/users/bf00d295eb51459494312eb2f228bf83/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.696969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790b04f-da65-4d69-a67b-fded7b8493aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=133912be27ab446b9194a7d75dc6475f, user_id=bf00d295eb51459494312eb2f228bf83, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.699642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790b04f-da65-4d69-a67b-fded7b8493aa admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.714677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1180/2341] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:54 2022] PUT /identity/v3/projects/133912be27ab446b9194a7d75dc6475f/users/bf00d295eb51459494312eb2f228bf83/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:54.718724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790b04f-da65-4d69-a67b-fded7b8493aa admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.720390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f790b04f-da65-4d69-a67b-fded7b8493aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:54.721945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497cc11-2b1b-450a-a61a-45ec5679402e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.726774 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:54.782224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:54.789721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.790808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.791693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.792076 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] PATH_INFO: `/v3/users/6a755398f54543be89f2a003fe163c5d` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.793601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a755398f54543be89f2a003fe163c5d)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.794347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790b04f-da65-4d69-a67b-fded7b8493aa admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.796174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.799135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a535b-4cea-4f03-ae87-a7ce4a260b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.799816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a535b-4cea-4f03-ae87-a7ce4a260b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.800104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a535b-4cea-4f03-ae87-a7ce4a260b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.800384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a535b-4cea-4f03-ae87-a7ce4a260b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.801293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a535b-4cea-4f03-ae87-a7ce4a260b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.802889 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a535b-4cea-4f03-ae87-a7ce4a260b79 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.823213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1181/2342] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:54.829413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a535b-4cea-4f03-ae87-a7ce4a260b79 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.843396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a535b-4cea-4f03-ae87-a7ce4a260b79 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.851149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb41ef9a-c31a-4d35-8a49-54e93fe2257c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.851764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41ef9a-c31a-4d35-8a49-54e93fe2257c admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.852127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41ef9a-c31a-4d35-8a49-54e93fe2257c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.852840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41ef9a-c31a-4d35-8a49-54e93fe2257c admin admin] PATH_INFO: `/v3/projects/133912be27ab446b9194a7d75dc6475f/users/bf00d295eb51459494312eb2f228bf83/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.854659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41ef9a-c31a-4d35-8a49-54e93fe2257c admin admin] RBAC: Authorizing `identity:create_grant(project_id=133912be27ab446b9194a7d75dc6475f, user_id=bf00d295eb51459494312eb2f228bf83, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.857768 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb41ef9a-c31a-4d35-8a49-54e93fe2257c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.867587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a755398f54543be89f2a003fe163c5d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:54.875251 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a755398f54543be89f2a003fe163c5d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:54.877259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1182/2343] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:54 2022] PUT /identity/v3/projects/133912be27ab446b9194a7d75dc6475f/users/bf00d295eb51459494312eb2f228bf83/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:54.888186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0a96e-59ff-4873-a798-8019c23af095 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.888517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0a96e-59ff-4873-a798-8019c23af095 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.888848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0a96e-59ff-4873-a798-8019c23af095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.902548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a755398f54543be89f2a003fe163c5d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:54.914969 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a755398f54543be89f2a003fe163c5d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:54.916250 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a755398f54543be89f2a003fe163c5d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:54.921972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-23e0a96e-59ff-4873-a798-8019c23af095 None None] MFA Rules not processed for user `bf00d295eb51459494312eb2f228bf83`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:54.923211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-23e0a96e-59ff-4873-a798-8019c23af095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:54.923691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1162/2344] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:54 2022] DELETE /identity/v3/users/6a755398f54543be89f2a003fe163c5d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:55.047813 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1183/2345] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:55.066625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.066923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:55.156237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dd166-94c3-40fc-b403-bbcc0fa3be16 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.161423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86f83f-ef7d-4978-8f5d-15ee90adb345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.161423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86f83f-ef7d-4978-8f5d-15ee90adb345 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.161881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86f83f-ef7d-4978-8f5d-15ee90adb345 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.162228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86f83f-ef7d-4978-8f5d-15ee90adb345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.174104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86f83f-ef7d-4978-8f5d-15ee90adb345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.175912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86f83f-ef7d-4978-8f5d-15ee90adb345 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.188787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1163/2346] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:35:55.376429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0a96e-59ff-4873-a798-8019c23af095 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.377945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-23e0a96e-59ff-4873-a798-8019c23af095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:55.471327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83037be6-d912-4ace-8401-195571fff564 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.471327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83037be6-d912-4ace-8401-195571fff564 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.471327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83037be6-d912-4ace-8401-195571fff564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.492151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-83037be6-d912-4ace-8401-195571fff564 None None] MFA Rules not processed for user `35f252f0338547aea73d1eb7c81ca199`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:55.493496 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-83037be6-d912-4ace-8401-195571fff564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:55.495694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0a96e-59ff-4873-a798-8019c23af095 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.502333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.503211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.503574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.503753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] PATH_INFO: `/v3/projects/39b6f4b1689c41349866eb0d627b386a` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.505220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] RBAC: Authorizing `identity:delete_project(project_id=39b6f4b1689c41349866eb0d627b386a)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.508166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.532394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:55.589099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1164/2347] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:55.599378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39b6f4b1689c41349866eb0d627b386a', 'request_id': 'req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:55.614378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39b6f4b1689c41349866eb0d627b386a', 'request_id': 'req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:55.627595 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39b6f4b1689c41349866eb0d627b386a', 'request_id': 'req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:55.654443 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] The token cache is being invalidate because project 39b6f4b1689c41349866eb0d627b386a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:55.654970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:55.656169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:55.665408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1184/2348] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:55 2022] DELETE /identity/v3/projects/39b6f4b1689c41349866eb0d627b386a => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:55.666649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd57fa2-fd7e-4141-baa5-785d27ec8442 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.666956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd57fa2-fd7e-4141-baa5-785d27ec8442 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.667200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd57fa2-fd7e-4141-baa5-785d27ec8442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.700154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6fd57fa2-fd7e-4141-baa5-785d27ec8442 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:55.702431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6fd57fa2-fd7e-4141-baa5-785d27ec8442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:55.744336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:55.809739 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1165/2349] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:55.812039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.814387 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:55.818430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83065dc-89b7-405d-a07f-b7b32302fc93 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.818850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83065dc-89b7-405d-a07f-b7b32302fc93 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.818984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83065dc-89b7-405d-a07f-b7b32302fc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.837492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a5e81-decd-4bd5-9ab8-a7f1ec75964d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.840014 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f83065dc-89b7-405d-a07f-b7b32302fc93 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:55.841281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f83065dc-89b7-405d-a07f-b7b32302fc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:55.843417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec2891-5f62-46ee-839c-e195ee116da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.844315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec2891-5f62-46ee-839c-e195ee116da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.844630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec2891-5f62-46ee-839c-e195ee116da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.844911 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec2891-5f62-46ee-839c-e195ee116da8 admin admin] PATH_INFO: `/v3/users/03cdb5e029f64dcf976b8dc2eb7fc8f2` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.846114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec2891-5f62-46ee-839c-e195ee116da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03cdb5e029f64dcf976b8dc2eb7fc8f2)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.848024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec2891-5f62-46ee-839c-e195ee116da8 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.849392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1166/2350] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:55.854735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83065dc-89b7-405d-a07f-b7b32302fc93 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.867951 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83065dc-89b7-405d-a07f-b7b32302fc93 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.873532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01229245-5572-48e0-9289-bd521410f993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.874404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01229245-5572-48e0-9289-bd521410f993 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.874712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01229245-5572-48e0-9289-bd521410f993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.875063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01229245-5572-48e0-9289-bd521410f993 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.875878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01229245-5572-48e0-9289-bd521410f993 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:55.876260 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01229245-5572-48e0-9289-bd521410f993 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.878367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01229245-5572-48e0-9289-bd521410f993 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.896358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1167/2351] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:55.897796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01229245-5572-48e0-9289-bd521410f993 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.910573 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01229245-5572-48e0-9289-bd521410f993 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.916498 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b5ec2891-5f62-46ee-839c-e195ee116da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03cdb5e029f64dcf976b8dc2eb7fc8f2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:55.922845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a873b-0b8e-425b-90da-4ae48324e16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.923911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a873b-0b8e-425b-90da-4ae48324e16b admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.924573 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a873b-0b8e-425b-90da-4ae48324e16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.924943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a873b-0b8e-425b-90da-4ae48324e16b admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.925911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a873b-0b8e-425b-90da-4ae48324e16b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.926939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b5ec2891-5f62-46ee-839c-e195ee116da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03cdb5e029f64dcf976b8dc2eb7fc8f2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:55.928330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b5ec2891-5f62-46ee-839c-e195ee116da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03cdb5e029f64dcf976b8dc2eb7fc8f2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:55.930233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a873b-0b8e-425b-90da-4ae48324e16b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.939017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b5ec2891-5f62-46ee-839c-e195ee116da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03cdb5e029f64dcf976b8dc2eb7fc8f2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:55.939819 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b5ec2891-5f62-46ee-839c-e195ee116da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03cdb5e029f64dcf976b8dc2eb7fc8f2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:55.960973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1185/2352] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:55 2022] DELETE /identity/v3/users/03cdb5e029f64dcf976b8dc2eb7fc8f2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:55.965842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98640ce-bd40-486a-bd7a-c722d3878d28 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.966280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98640ce-bd40-486a-bd7a-c722d3878d28 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.966585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98640ce-bd40-486a-bd7a-c722d3878d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.966943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1168/2353] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:55 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:55.971180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a873b-0b8e-425b-90da-4ae48324e16b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.973114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3b4a873b-0b8e-425b-90da-4ae48324e16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:55.986288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-f98640ce-bd40-486a-bd7a-c722d3878d28 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:55.987952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f98640ce-bd40-486a-bd7a-c722d3878d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:56.065499 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a873b-0b8e-425b-90da-4ae48324e16b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.066201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1186/2354] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:56.075456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940edb7-7228-4f34-bca2-0b57e1717336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.076590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940edb7-7228-4f34-bca2-0b57e1717336 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.077001 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940edb7-7228-4f34-bca2-0b57e1717336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.077122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98640ce-bd40-486a-bd7a-c722d3878d28 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:56.077635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940edb7-7228-4f34-bca2-0b57e1717336 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.078426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940edb7-7228-4f34-bca2-0b57e1717336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.078535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f98640ce-bd40-486a-bd7a-c722d3878d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:56.080611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940edb7-7228-4f34-bca2-0b57e1717336 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.103036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:56.111125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1169/2355] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:55 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:56.112674 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940edb7-7228-4f34-bca2-0b57e1717336 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:56.125963 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940edb7-7228-4f34-bca2-0b57e1717336 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.138800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca157415-9531-4405-b771-3501c127c43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.139397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca157415-9531-4405-b771-3501c127c43e admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.139684 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca157415-9531-4405-b771-3501c127c43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.139996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca157415-9531-4405-b771-3501c127c43e admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.140769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca157415-9531-4405-b771-3501c127c43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.142520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca157415-9531-4405-b771-3501c127c43e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.153517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98640ce-bd40-486a-bd7a-c722d3878d28 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.156065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1170/2356] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:56.157923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca182b07-372f-457d-a372-2cee52f68aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.158638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca182b07-372f-457d-a372-2cee52f68aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.158930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca182b07-372f-457d-a372-2cee52f68aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.159214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca182b07-372f-457d-a372-2cee52f68aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.160847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca157415-9531-4405-b771-3501c127c43e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:56.171007 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca182b07-372f-457d-a372-2cee52f68aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.172756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca182b07-372f-457d-a372-2cee52f68aa5 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.174312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca157415-9531-4405-b771-3501c127c43e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.183641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc8851-1353-4566-8d6d-4dc60811012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.184546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc8851-1353-4566-8d6d-4dc60811012e admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.184841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc8851-1353-4566-8d6d-4dc60811012e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.185117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc8851-1353-4566-8d6d-4dc60811012e admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.185864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc8851-1353-4566-8d6d-4dc60811012e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.186182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1187/2357] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:56.188163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc8851-1353-4566-8d6d-4dc60811012e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.213780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1171/2358] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:56.215286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca182b07-372f-457d-a372-2cee52f68aa5 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:56.226590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca182b07-372f-457d-a372-2cee52f68aa5 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.232911 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6acf8f-7172-4245-ae2f-dc1a9c85b9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.233720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6acf8f-7172-4245-ae2f-dc1a9c85b9be admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.234081 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6acf8f-7172-4245-ae2f-dc1a9c85b9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.234442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6acf8f-7172-4245-ae2f-dc1a9c85b9be admin admin] PATH_INFO: `/v3/projects/6e75fae58f7348c48d7f349baa789366/users/7155a064efa8470eacd810ee6627347e/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.259902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6acf8f-7172-4245-ae2f-dc1a9c85b9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e75fae58f7348c48d7f349baa789366, user_id=7155a064efa8470eacd810ee6627347e, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.263425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6acf8f-7172-4245-ae2f-dc1a9c85b9be admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.289965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1188/2359] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:56 2022] PUT /identity/v3/projects/6e75fae58f7348c48d7f349baa789366/users/7155a064efa8470eacd810ee6627347e/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:56.303686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679413d9-1e63-41cf-b8d7-6ab77c43531d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.304689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679413d9-1e63-41cf-b8d7-6ab77c43531d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.305079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679413d9-1e63-41cf-b8d7-6ab77c43531d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.358945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-679413d9-1e63-41cf-b8d7-6ab77c43531d None None] MFA Rules not processed for user `7155a064efa8470eacd810ee6627347e`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:56.360182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-679413d9-1e63-41cf-b8d7-6ab77c43531d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:56.450945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:56.486618 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1172/2360] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:56.509022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6acf8f-7172-4245-ae2f-dc1a9c85b9be admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:56.510857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3a6acf8f-7172-4245-ae2f-dc1a9c85b9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:56.684915 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6acf8f-7172-4245-ae2f-dc1a9c85b9be admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.690797 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4a5e5-9bcd-48eb-a65f-2bb1ab9133c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.691537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a5e5-9bcd-48eb-a65f-2bb1ab9133c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.692213 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a5e5-9bcd-48eb-a65f-2bb1ab9133c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.693098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a5e5-9bcd-48eb-a65f-2bb1ab9133c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.721173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4a5e5-9bcd-48eb-a65f-2bb1ab9133c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.725319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4a5e5-9bcd-48eb-a65f-2bb1ab9133c7 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.759827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679413d9-1e63-41cf-b8d7-6ab77c43531d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:56.761393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-679413d9-1e63-41cf-b8d7-6ab77c43531d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:56.774782 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1189/2361] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:35:56.879239 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679413d9-1e63-41cf-b8d7-6ab77c43531d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.885419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.886441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.886853 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.887205 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] PATH_INFO: `/v3/projects/feefb78f565b435fb4731d897ba0ddef` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.888576 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] RBAC: Authorizing `identity:delete_project(project_id=feefb78f565b435fb4731d897ba0ddef)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.890659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.969467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feefb78f565b435fb4731d897ba0ddef', 'request_id': 'req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:56.979539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feefb78f565b435fb4731d897ba0ddef', 'request_id': 'req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:56.993344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feefb78f565b435fb4731d897ba0ddef', 'request_id': 'req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.012720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] The token cache is being invalidate because project feefb78f565b435fb4731d897ba0ddef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:57.013108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.013792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.015712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1173/2362] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:56 2022] DELETE /identity/v3/projects/feefb78f565b435fb4731d897ba0ddef => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:57.019674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4a5e5-9bcd-48eb-a65f-2bb1ab9133c7 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:57.030947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4a5e5-9bcd-48eb-a65f-2bb1ab9133c7 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:57.103592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f7ab9-eaf0-4a44-82f7-32e2317bb970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:57.104638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f7ab9-eaf0-4a44-82f7-32e2317bb970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:57.104924 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f7ab9-eaf0-4a44-82f7-32e2317bb970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:57.105218 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f7ab9-eaf0-4a44-82f7-32e2317bb970 admin admin] PATH_INFO: `/v3/users/bf00d295eb51459494312eb2f228bf83` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:57.106451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f7ab9-eaf0-4a44-82f7-32e2317bb970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf00d295eb51459494312eb2f228bf83)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:57.108648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f7ab9-eaf0-4a44-82f7-32e2317bb970 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:57.182086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:57.191867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6b2f7ab9-eaf0-4a44-82f7-32e2317bb970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf00d295eb51459494312eb2f228bf83'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.214199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6b2f7ab9-eaf0-4a44-82f7-32e2317bb970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf00d295eb51459494312eb2f228bf83'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.214199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6b2f7ab9-eaf0-4a44-82f7-32e2317bb970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf00d295eb51459494312eb2f228bf83'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.223328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6b2f7ab9-eaf0-4a44-82f7-32e2317bb970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf00d295eb51459494312eb2f228bf83'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.224340 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6b2f7ab9-eaf0-4a44-82f7-32e2317bb970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf00d295eb51459494312eb2f228bf83'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.281523 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1190/2363] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:35:57 2022] DELETE /identity/v3/users/bf00d295eb51459494312eb2f228bf83 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:57.622026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:57.627100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:57.652787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285d0d9-d3e7-4fa0-b4c0-b6c10358b7de admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:57.769671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95800880-78cc-43ab-a616-9aac778788da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:57.770970 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95800880-78cc-43ab-a616-9aac778788da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:57.774830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95800880-78cc-43ab-a616-9aac778788da admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:57.774830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95800880-78cc-43ab-a616-9aac778788da admin admin] PATH_INFO: `/v3/projects/133912be27ab446b9194a7d75dc6475f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:57.803263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95800880-78cc-43ab-a616-9aac778788da admin admin] RBAC: Authorizing `identity:delete_project(project_id=133912be27ab446b9194a7d75dc6475f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:57.806108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95800880-78cc-43ab-a616-9aac778788da admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:57.821613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:57.878839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-95800880-78cc-43ab-a616-9aac778788da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '133912be27ab446b9194a7d75dc6475f', 'request_id': 'req-95800880-78cc-43ab-a616-9aac778788da'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.887982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-95800880-78cc-43ab-a616-9aac778788da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '133912be27ab446b9194a7d75dc6475f', 'request_id': 'req-95800880-78cc-43ab-a616-9aac778788da'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.901037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-95800880-78cc-43ab-a616-9aac778788da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '133912be27ab446b9194a7d75dc6475f', 'request_id': 'req-95800880-78cc-43ab-a616-9aac778788da'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.919789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-95800880-78cc-43ab-a616-9aac778788da admin admin] The token cache is being invalidate because project 133912be27ab446b9194a7d75dc6475f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:35:57.920463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-95800880-78cc-43ab-a616-9aac778788da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.921202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-95800880-78cc-43ab-a616-9aac778788da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:35:57.922819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1174/2364] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:35:57 2022] DELETE /identity/v3/projects/133912be27ab446b9194a7d75dc6475f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:57.928865 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5536c-9054-4622-98bd-1d8a29f88bce None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:57.929350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5536c-9054-4622-98bd-1d8a29f88bce None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:57.929761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5536c-9054-4622-98bd-1d8a29f88bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:57.950665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-c4f5536c-9054-4622-98bd-1d8a29f88bce None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:57.952011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c4f5536c-9054-4622-98bd-1d8a29f88bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:58.084521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1191/2365] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:58.097518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431d732-66e9-4442-9aeb-0ba438741a7d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.097994 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431d732-66e9-4442-9aeb-0ba438741a7d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.098461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431d732-66e9-4442-9aeb-0ba438741a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.126594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1431d732-66e9-4442-9aeb-0ba438741a7d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:58.129319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1431d732-66e9-4442-9aeb-0ba438741a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:58.144533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1175/2366] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:35:58.149098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5536c-9054-4622-98bd-1d8a29f88bce None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:58.160771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5536c-9054-4622-98bd-1d8a29f88bce None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:58.166500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c805105-411b-4a10-96e4-6846eab90cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:58.167359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c805105-411b-4a10-96e4-6846eab90cce admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.167645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c805105-411b-4a10-96e4-6846eab90cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.167926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c805105-411b-4a10-96e4-6846eab90cce admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.168902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c805105-411b-4a10-96e4-6846eab90cce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:58.169220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c805105-411b-4a10-96e4-6846eab90cce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:58.171202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c805105-411b-4a10-96e4-6846eab90cce admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:58.190768 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1192/2367] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:58.195867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431d732-66e9-4442-9aeb-0ba438741a7d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:58.207698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431d732-66e9-4442-9aeb-0ba438741a7d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:58.215330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553aa64f-3857-49c0-a351-940c272f775c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:58.216057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553aa64f-3857-49c0-a351-940c272f775c admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.216416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553aa64f-3857-49c0-a351-940c272f775c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.216805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553aa64f-3857-49c0-a351-940c272f775c admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.217589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553aa64f-3857-49c0-a351-940c272f775c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:58.219675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553aa64f-3857-49c0-a351-940c272f775c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:58.269520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1176/2368] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:35:58 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:58.271362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c805105-411b-4a10-96e4-6846eab90cce admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:58.293714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c805105-411b-4a10-96e4-6846eab90cce admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:58.400123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a56b74-8043-4a27-8fa7-44399f140ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:58.402063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a56b74-8043-4a27-8fa7-44399f140ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.402675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a56b74-8043-4a27-8fa7-44399f140ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.403165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a56b74-8043-4a27-8fa7-44399f140ef1 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.404947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a56b74-8043-4a27-8fa7-44399f140ef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:58.407814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a56b74-8043-4a27-8fa7-44399f140ef1 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:58.454140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1193/2369] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:35:58 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:58.459721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553aa64f-3857-49c0-a351-940c272f775c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:58.479082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553aa64f-3857-49c0-a351-940c272f775c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:58.487787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7062f-f159-4338-9212-103471233cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:58.489288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7062f-f159-4338-9212-103471233cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.489902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7062f-f159-4338-9212-103471233cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.490401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7062f-f159-4338-9212-103471233cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.492387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7062f-f159-4338-9212-103471233cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:58.494996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7062f-f159-4338-9212-103471233cc6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:58.512554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1177/2370] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:35:58.515656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a56b74-8043-4a27-8fa7-44399f140ef1 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:58.527648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a56b74-8043-4a27-8fa7-44399f140ef1 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:58.541969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0daf400-6aa9-4054-ac23-dc7fb3ffb1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:58.542818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daf400-6aa9-4054-ac23-dc7fb3ffb1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.543207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daf400-6aa9-4054-ac23-dc7fb3ffb1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.543598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daf400-6aa9-4054-ac23-dc7fb3ffb1e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.544443 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0daf400-6aa9-4054-ac23-dc7fb3ffb1e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:58.546233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0daf400-6aa9-4054-ac23-dc7fb3ffb1e1 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:58.559468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1194/2371] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:35:58.564986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7062f-f159-4338-9212-103471233cc6 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:58.576312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7062f-f159-4338-9212-103471233cc6 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:58.581406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b05ad-7597-4fcf-941b-78bf99eed67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:58.583007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b05ad-7597-4fcf-941b-78bf99eed67b admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.583299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b05ad-7597-4fcf-941b-78bf99eed67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.583659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b05ad-7597-4fcf-941b-78bf99eed67b admin admin] PATH_INFO: `/v3/projects/c96d401f97834a2586165d4e7e0eef70/users/b3d91f1bf80c4003976d7ebd84170c65/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.611016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b05ad-7597-4fcf-941b-78bf99eed67b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96d401f97834a2586165d4e7e0eef70, user_id=b3d91f1bf80c4003976d7ebd84170c65, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:58.611016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b05ad-7597-4fcf-941b-78bf99eed67b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:58.638906 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1178/2372] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:35:58 2022] PUT /identity/v3/projects/c96d401f97834a2586165d4e7e0eef70/users/b3d91f1bf80c4003976d7ebd84170c65/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:58.645106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e79d9e-94d5-49fb-b721-ea468cd2b210 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.645708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e79d9e-94d5-49fb-b721-ea468cd2b210 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.646306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e79d9e-94d5-49fb-b721-ea468cd2b210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.690750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-69e79d9e-94d5-49fb-b721-ea468cd2b210 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:58.692113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-69e79d9e-94d5-49fb-b721-ea468cd2b210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:58.715058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:58.783373 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1195/2373] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:35:58.798123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b05ad-7597-4fcf-941b-78bf99eed67b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:58.800655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-716b05ad-7597-4fcf-941b-78bf99eed67b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:58.916726 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b05ad-7597-4fcf-941b-78bf99eed67b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:58.921181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d2e2e-7b48-41fc-84f7-ce2ab676cabf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:58.922006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d2e2e-7b48-41fc-84f7-ce2ab676cabf service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.922298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d2e2e-7b48-41fc-84f7-ce2ab676cabf service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.922579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d2e2e-7b48-41fc-84f7-ce2ab676cabf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.936441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d2e2e-7b48-41fc-84f7-ce2ab676cabf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:58.937960 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d2e2e-7b48-41fc-84f7-ce2ab676cabf service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:58.949982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1179/2374] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:36:00.071475 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d877e-8420-4f70-bd53-fa64468f661d None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.072283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d877e-8420-4f70-bd53-fa64468f661d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.072283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d877e-8420-4f70-bd53-fa64468f661d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.114168 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-750d877e-8420-4f70-bd53-fa64468f661d None None] MFA Rules not processed for user `b3d91f1bf80c4003976d7ebd84170c65`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:00.115978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-750d877e-8420-4f70-bd53-fa64468f661d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:00.245737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1196/2375] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:00.247144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d2e2e-7b48-41fc-84f7-ce2ab676cabf service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.266435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d2e2e-7b48-41fc-84f7-ce2ab676cabf service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.285213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb127d3e-025b-4ec7-86a3-b45927cc3b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.286833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb127d3e-025b-4ec7-86a3-b45927cc3b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.287552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb127d3e-025b-4ec7-86a3-b45927cc3b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.289090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb127d3e-025b-4ec7-86a3-b45927cc3b2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.290689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb127d3e-025b-4ec7-86a3-b45927cc3b2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.294631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb127d3e-025b-4ec7-86a3-b45927cc3b2b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.339592 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1180/2376] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:36:00 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:00.344388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d877e-8420-4f70-bd53-fa64468f661d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.388393 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d877e-8420-4f70-bd53-fa64468f661d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.467331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e74f69-1afa-4e26-a35d-eb45bc5b970d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.468535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e74f69-1afa-4e26-a35d-eb45bc5b970d admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.469027 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e74f69-1afa-4e26-a35d-eb45bc5b970d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.469392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e74f69-1afa-4e26-a35d-eb45bc5b970d admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.470374 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e74f69-1afa-4e26-a35d-eb45bc5b970d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.473711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e74f69-1afa-4e26-a35d-eb45bc5b970d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.507360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1197/2377] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:36:00 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:00.513818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb127d3e-025b-4ec7-86a3-b45927cc3b2b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.550645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb127d3e-025b-4ec7-86a3-b45927cc3b2b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.556231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e74f69-1afa-4e26-a35d-eb45bc5b970d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.558623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-72e74f69-1afa-4e26-a35d-eb45bc5b970d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:00.561513 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532658e4-efbe-4f8e-8748-cf1fc00e375e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.562858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532658e4-efbe-4f8e-8748-cf1fc00e375e admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.563317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532658e4-efbe-4f8e-8748-cf1fc00e375e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.563815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532658e4-efbe-4f8e-8748-cf1fc00e375e admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.565066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532658e4-efbe-4f8e-8748-cf1fc00e375e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.567666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532658e4-efbe-4f8e-8748-cf1fc00e375e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.573838 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e74f69-1afa-4e26-a35d-eb45bc5b970d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.581488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467f109-6342-4f8d-a5ec-f17a4793b053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.582384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467f109-6342-4f8d-a5ec-f17a4793b053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.582672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467f109-6342-4f8d-a5ec-f17a4793b053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.582965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467f109-6342-4f8d-a5ec-f17a4793b053 admin admin] PATH_INFO: `/v3/users/7155a064efa8470eacd810ee6627347e` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.584270 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467f109-6342-4f8d-a5ec-f17a4793b053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7155a064efa8470eacd810ee6627347e)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.587237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467f109-6342-4f8d-a5ec-f17a4793b053 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.589463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1181/2378] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:36:00.591154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532658e4-efbe-4f8e-8748-cf1fc00e375e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.607168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532658e4-efbe-4f8e-8748-cf1fc00e375e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.618263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a42207-8aea-446c-8bed-7b5488b6e103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.626245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a42207-8aea-446c-8bed-7b5488b6e103 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.626573 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a42207-8aea-446c-8bed-7b5488b6e103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.626768 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a42207-8aea-446c-8bed-7b5488b6e103 admin admin] PATH_INFO: `/v3/projects/fa22f60be7534926abf464eba1c17f5e/users/56c55e99c31342c3af75a3a8f98cab54/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.631032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:00.654907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a42207-8aea-446c-8bed-7b5488b6e103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa22f60be7534926abf464eba1c17f5e, user_id=56c55e99c31342c3af75a3a8f98cab54, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.657645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a42207-8aea-446c-8bed-7b5488b6e103 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.660184 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:00.687402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7467f109-6342-4f8d-a5ec-f17a4793b053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7155a064efa8470eacd810ee6627347e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:00.693166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1182/2379] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:36:00 2022] PUT /identity/v3/projects/fa22f60be7534926abf464eba1c17f5e/users/56c55e99c31342c3af75a3a8f98cab54/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:00.699103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a42207-8aea-446c-8bed-7b5488b6e103 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.719209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7467f109-6342-4f8d-a5ec-f17a4793b053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7155a064efa8470eacd810ee6627347e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:00.720987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7467f109-6342-4f8d-a5ec-f17a4793b053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7155a064efa8470eacd810ee6627347e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:00.737402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7467f109-6342-4f8d-a5ec-f17a4793b053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7155a064efa8470eacd810ee6627347e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:00.738628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7467f109-6342-4f8d-a5ec-f17a4793b053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7155a064efa8470eacd810ee6627347e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:00.776304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1198/2380] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:36:00 2022] DELETE /identity/v3/users/7155a064efa8470eacd810ee6627347e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:36:00.778189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1f927-f4da-4cc2-870d-f39f4397815b None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.778642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1f927-f4da-4cc2-870d-f39f4397815b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.779088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1f927-f4da-4cc2-870d-f39f4397815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.818189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-e6e1f927-f4da-4cc2-870d-f39f4397815b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:00.821411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-e6e1f927-f4da-4cc2-870d-f39f4397815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:00.823640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a42207-8aea-446c-8bed-7b5488b6e103 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.920093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66e080-0ea7-4cc7-ac4d-a264d1827ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.923633 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66e080-0ea7-4cc7-ac4d-a264d1827ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.924724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66e080-0ea7-4cc7-ac4d-a264d1827ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.925233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66e080-0ea7-4cc7-ac4d-a264d1827ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.927094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66e080-0ea7-4cc7-ac4d-a264d1827ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.930314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66e080-0ea7-4cc7-ac4d-a264d1827ca1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.951911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1183/2381] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:36:00.957587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66e080-0ea7-4cc7-ac4d-a264d1827ca1 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.969786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-eb66e080-0ea7-4cc7-ac4d-a264d1827ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:00.971983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1199/2382] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:00.985891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66e080-0ea7-4cc7-ac4d-a264d1827ca1 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.992799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730fddb-c1fe-4128-bdcf-23c30f6a3115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:01.000956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730fddb-c1fe-4128-bdcf-23c30f6a3115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:01.001437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730fddb-c1fe-4128-bdcf-23c30f6a3115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:01.001763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730fddb-c1fe-4128-bdcf-23c30f6a3115 admin admin] PATH_INFO: `/v3/projects/fa22f60be7534926abf464eba1c17f5e/users/56c55e99c31342c3af75a3a8f98cab54/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:01.005357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1f927-f4da-4cc2-870d-f39f4397815b None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:01.009629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730fddb-c1fe-4128-bdcf-23c30f6a3115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa22f60be7534926abf464eba1c17f5e, user_id=56c55e99c31342c3af75a3a8f98cab54, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:01.011612 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-e6e1f927-f4da-4cc2-870d-f39f4397815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:01.014388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730fddb-c1fe-4128-bdcf-23c30f6a3115 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:01.041800 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1184/2383] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:36:00 2022] PUT /identity/v3/projects/fa22f60be7534926abf464eba1c17f5e/users/56c55e99c31342c3af75a3a8f98cab54/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:01.166952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1f927-f4da-4cc2-870d-f39f4397815b None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:01.247347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da05446-ebf0-4772-8b4f-d4da62aa1ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:01.248509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da05446-ebf0-4772-8b4f-d4da62aa1ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:01.249140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da05446-ebf0-4772-8b4f-d4da62aa1ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:01.249711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da05446-ebf0-4772-8b4f-d4da62aa1ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:01.296059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:01.391424 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da05446-ebf0-4772-8b4f-d4da62aa1ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:01.395011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da05446-ebf0-4772-8b4f-d4da62aa1ec1 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:01.409059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1200/2384] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:02.005667 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730fddb-c1fe-4128-bdcf-23c30f6a3115 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.009167 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e730fddb-c1fe-4128-bdcf-23c30f6a3115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:02.024883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730fddb-c1fe-4128-bdcf-23c30f6a3115 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.031219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.032294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.032645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.032925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742 admin admin] PATH_INFO: `/v3/projects/6e75fae58f7348c48d7f349baa789366` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.034428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e75fae58f7348c48d7f349baa789366)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.036381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.124387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e75fae58f7348c48d7f349baa789366', 'request_id': 'req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:02.150434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e75fae58f7348c48d7f349baa789366', 'request_id': 'req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:02.182577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e75fae58f7348c48d7f349baa789366', 'request_id': 'req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:02.187624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:02.216232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742 admin admin] The token cache is being invalidate because project 6e75fae58f7348c48d7f349baa789366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:36:02.216655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:02.217479 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e96b5bd1-0887-4000-9bd3-47c1c1ca1742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:02.226194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1185/2385] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:36:02 2022] DELETE /identity/v3/projects/6e75fae58f7348c48d7f349baa789366 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:36:02.226952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387e193-5619-4c76-802a-807b0449ea4c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.226952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387e193-5619-4c76-802a-807b0449ea4c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.227988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387e193-5619-4c76-802a-807b0449ea4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.255338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-8387e193-5619-4c76-802a-807b0449ea4c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:02.257678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8387e193-5619-4c76-802a-807b0449ea4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:02.278925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94dcf9-a49f-4840-92fa-7c19cfd2aae1 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.279353 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94dcf9-a49f-4840-92fa-7c19cfd2aae1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.280918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94dcf9-a49f-4840-92fa-7c19cfd2aae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.326934 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6d94dcf9-a49f-4840-92fa-7c19cfd2aae1 None None] MFA Rules not processed for user `56c55e99c31342c3af75a3a8f98cab54`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:02.329449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6d94dcf9-a49f-4840-92fa-7c19cfd2aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:02.350184 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1201/2386] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:36:02.366570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ec3f5-b491-4d22-b549-df5c0f1aabd7 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.366973 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ec3f5-b491-4d22-b549-df5c0f1aabd7 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.368295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ec3f5-b491-4d22-b549-df5c0f1aabd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.432681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-070ec3f5-b491-4d22-b549-df5c0f1aabd7 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:02.437933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-070ec3f5-b491-4d22-b549-df5c0f1aabd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:02.458500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1202/2387] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:36:02.467476 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ec3f5-b491-4d22-b549-df5c0f1aabd7 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.484073 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ec3f5-b491-4d22-b549-df5c0f1aabd7 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.505458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d355d-0a7b-43e3-add9-37a798cbd167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.507009 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d355d-0a7b-43e3-add9-37a798cbd167 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.507313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d355d-0a7b-43e3-add9-37a798cbd167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.507596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d355d-0a7b-43e3-add9-37a798cbd167 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.508409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d355d-0a7b-43e3-add9-37a798cbd167 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:02.508704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d355d-0a7b-43e3-add9-37a798cbd167 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.520251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d355d-0a7b-43e3-add9-37a798cbd167 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.536119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1203/2388] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:02.541392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d355d-0a7b-43e3-add9-37a798cbd167 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.555114 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d355d-0a7b-43e3-add9-37a798cbd167 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.562955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643fe9f-04e0-4250-8dbe-02f29aec6c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.564135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643fe9f-04e0-4250-8dbe-02f29aec6c96 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.564135 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643fe9f-04e0-4250-8dbe-02f29aec6c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.564566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643fe9f-04e0-4250-8dbe-02f29aec6c96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.565910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643fe9f-04e0-4250-8dbe-02f29aec6c96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.568909 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643fe9f-04e0-4250-8dbe-02f29aec6c96 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.574623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1186/2389] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:02.588900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94dcf9-a49f-4840-92fa-7c19cfd2aae1 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.590520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6d94dcf9-a49f-4840-92fa-7c19cfd2aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:02.646846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1204/2390] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:36:02 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:02.647285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643fe9f-04e0-4250-8dbe-02f29aec6c96 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.696714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94dcf9-a49f-4840-92fa-7c19cfd2aae1 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.734257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643fe9f-04e0-4250-8dbe-02f29aec6c96 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.740880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d98ad-c291-4865-afa1-7bd71861e001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.741590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d98ad-c291-4865-afa1-7bd71861e001 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.741979 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d98ad-c291-4865-afa1-7bd71861e001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.742272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d98ad-c291-4865-afa1-7bd71861e001 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.742983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d98ad-c291-4865-afa1-7bd71861e001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.744996 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d98ad-c291-4865-afa1-7bd71861e001 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.764124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0139c49-261f-4026-8631-fdf9b7100a68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.765259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0139c49-261f-4026-8631-fdf9b7100a68 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.765637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0139c49-261f-4026-8631-fdf9b7100a68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.765921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0139c49-261f-4026-8631-fdf9b7100a68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.768368 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e0139c49-261f-4026-8631-fdf9b7100a68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:02.782227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1205/2391] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:36:02 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:02.786687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d98ad-c291-4865-afa1-7bd71861e001 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.798042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d98ad-c291-4865-afa1-7bd71861e001 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.803164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8836dee-bbea-4748-92e3-2785ee851d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.804456 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8836dee-bbea-4748-92e3-2785ee851d17 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.804861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8836dee-bbea-4748-92e3-2785ee851d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.805125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8836dee-bbea-4748-92e3-2785ee851d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.805964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8836dee-bbea-4748-92e3-2785ee851d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.807851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8836dee-bbea-4748-92e3-2785ee851d17 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.810197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:02.825330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1206/2392] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:36:02.827182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8836dee-bbea-4748-92e3-2785ee851d17 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.829077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:02.840104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8836dee-bbea-4748-92e3-2785ee851d17 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.840694 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0139c49-261f-4026-8631-fdf9b7100a68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.842179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0139c49-261f-4026-8631-fdf9b7100a68 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.848107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c505df-d5ab-40eb-a08d-3d9efdf0ee80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.849455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c505df-d5ab-40eb-a08d-3d9efdf0ee80 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.849954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c505df-d5ab-40eb-a08d-3d9efdf0ee80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.850530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c505df-d5ab-40eb-a08d-3d9efdf0ee80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.851902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c505df-d5ab-40eb-a08d-3d9efdf0ee80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.852444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1187/2393] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:36:02.855065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c505df-d5ab-40eb-a08d-3d9efdf0ee80 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.869387 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1207/2394] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:36:02.874056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0139c49-261f-4026-8631-fdf9b7100a68 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.887039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0139c49-261f-4026-8631-fdf9b7100a68 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.892430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2474fa-210d-4cde-ac1d-5a31ba80add5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.893502 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2474fa-210d-4cde-ac1d-5a31ba80add5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.893858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2474fa-210d-4cde-ac1d-5a31ba80add5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.894145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2474fa-210d-4cde-ac1d-5a31ba80add5 admin admin] PATH_INFO: `/v3/projects/bfa826fecf0d44ad9f5a23c9c83c15ab/users/2ffdc061c0c746cd844cf5e8aef68b81/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.911891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2474fa-210d-4cde-ac1d-5a31ba80add5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa826fecf0d44ad9f5a23c9c83c15ab, user_id=2ffdc061c0c746cd844cf5e8aef68b81, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.915008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2474fa-210d-4cde-ac1d-5a31ba80add5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.941196 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1188/2395] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:36:02 2022] PUT /identity/v3/projects/bfa826fecf0d44ad9f5a23c9c83c15ab/users/2ffdc061c0c746cd844cf5e8aef68b81/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:02.958018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2128d58-b73d-41a7-97b0-44239f5b3605 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.958309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2128d58-b73d-41a7-97b0-44239f5b3605 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.958663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2128d58-b73d-41a7-97b0-44239f5b3605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.988000 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-e2128d58-b73d-41a7-97b0-44239f5b3605 None None] MFA Rules not processed for user `2ffdc061c0c746cd844cf5e8aef68b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:02.989296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-e2128d58-b73d-41a7-97b0-44239f5b3605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:03.081510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1208/2396] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:03.093410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2474fa-210d-4cde-ac1d-5a31ba80add5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:03.095251 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ab2474fa-210d-4cde-ac1d-5a31ba80add5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:03.181340 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2474fa-210d-4cde-ac1d-5a31ba80add5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:03.185962 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0e211-3184-410c-8ee9-6754aa673bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:03.186905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0e211-3184-410c-8ee9-6754aa673bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:03.187316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0e211-3184-410c-8ee9-6754aa673bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:03.187589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0e211-3184-410c-8ee9-6754aa673bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:03.202857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0e211-3184-410c-8ee9-6754aa673bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:03.204754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0e211-3184-410c-8ee9-6754aa673bb8 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:03.221094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1189/2397] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:04.280650 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2128d58-b73d-41a7-97b0-44239f5b3605 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:04.291314 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2128d58-b73d-41a7-97b0-44239f5b3605 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:04.294945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8215e44-d88a-4353-9939-167cae517d58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:04.295606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8215e44-d88a-4353-9939-167cae517d58 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:04.295887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8215e44-d88a-4353-9939-167cae517d58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:04.296163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8215e44-d88a-4353-9939-167cae517d58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:04.364438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8215e44-d88a-4353-9939-167cae517d58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:04.366012 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8215e44-d88a-4353-9939-167cae517d58 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:04.376796 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1209/2398] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:13.214185 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:14.371730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=68 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:14.756296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e726e-91bd-4909-b9a9-85d7f8dcc459 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:14.757484 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e726e-91bd-4909-b9a9-85d7f8dcc459 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:14.758036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e726e-91bd-4909-b9a9-85d7f8dcc459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:14.777179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d87e726e-91bd-4909-b9a9-85d7f8dcc459 None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:14.778428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d87e726e-91bd-4909-b9a9-85d7f8dcc459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:14.846535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1190/2399] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 22:36:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:14.901518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8215e44-d88a-4353-9939-167cae517d58 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:14.903348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a8215e44-d88a-4353-9939-167cae517d58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:15.016736 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8215e44-d88a-4353-9939-167cae517d58 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:15.031554 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba58f2-81f3-449a-b2c3-2169fcd5b746 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:15.032505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba58f2-81f3-449a-b2c3-2169fcd5b746 service swift] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:15.032861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba58f2-81f3-449a-b2c3-2169fcd5b746 service swift] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:15.033218 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba58f2-81f3-449a-b2c3-2169fcd5b746 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:15.043236 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba58f2-81f3-449a-b2c3-2169fcd5b746 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:15.044867 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba58f2-81f3-449a-b2c3-2169fcd5b746 service swift] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:15.054559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1210/2400] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 22:36:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:36:16.171291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e726e-91bd-4909-b9a9-85d7f8dcc459 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:16.341327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e726e-91bd-4909-b9a9-85d7f8dcc459 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:16.354944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:16.356179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:16.356412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:16.356742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] PATH_INFO: `/v3/users/2ffdc061c0c746cd844cf5e8aef68b81` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:16.359163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ffdc061c0c746cd844cf5e8aef68b81)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:16.363571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:16.472639 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ffdc061c0c746cd844cf5e8aef68b81'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:16.481523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ffdc061c0c746cd844cf5e8aef68b81'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:16.517879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ffdc061c0c746cd844cf5e8aef68b81'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:16.556365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ffdc061c0c746cd844cf5e8aef68b81'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:16.561789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ffdc061c0c746cd844cf5e8aef68b81'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:16.572821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1191/2401] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:36:16 2022] DELETE /identity/v3/users/2ffdc061c0c746cd844cf5e8aef68b81 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:16.577219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a7677-53d9-42dc-93de-3a282e33181f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:16.577799 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a7677-53d9-42dc-93de-3a282e33181f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:16.577799 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a7677-53d9-42dc-93de-3a282e33181f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:16.598773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-403a7677-53d9-42dc-93de-3a282e33181f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:16.600709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-403a7677-53d9-42dc-93de-3a282e33181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:16.702756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1211/2402] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:16.710240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:16.711700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:16.772090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:16.824366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f5776-b983-4701-b7dc-6ea8e518f498 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:16.829281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40560820-9edf-4a5a-bb97-8f8a0f50e5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:16.829768 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40560820-9edf-4a5a-bb97-8f8a0f50e5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:16.830053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40560820-9edf-4a5a-bb97-8f8a0f50e5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:16.830806 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40560820-9edf-4a5a-bb97-8f8a0f50e5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:16.853971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40560820-9edf-4a5a-bb97-8f8a0f50e5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:16.856158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40560820-9edf-4a5a-bb97-8f8a0f50e5d7 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:16.874166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1192/2403] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:17.441980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a7677-53d9-42dc-93de-3a282e33181f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:17.446307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-403a7677-53d9-42dc-93de-3a282e33181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:17.522553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a7677-53d9-42dc-93de-3a282e33181f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:17.566487 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b6195-f2f4-485a-9970-388827a75cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:17.567560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b6195-f2f4-485a-9970-388827a75cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:17.567953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b6195-f2f4-485a-9970-388827a75cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:17.568928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b6195-f2f4-485a-9970-388827a75cb7 admin admin] PATH_INFO: `/v3/projects/bfa826fecf0d44ad9f5a23c9c83c15ab` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:17.574357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b6195-f2f4-485a-9970-388827a75cb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa826fecf0d44ad9f5a23c9c83c15ab)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:17.578453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b6195-f2f4-485a-9970-388827a75cb7 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:17.713447 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-863b6195-f2f4-485a-9970-388827a75cb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa826fecf0d44ad9f5a23c9c83c15ab', 'request_id': 'req-863b6195-f2f4-485a-9970-388827a75cb7'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:17.728390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-863b6195-f2f4-485a-9970-388827a75cb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa826fecf0d44ad9f5a23c9c83c15ab', 'request_id': 'req-863b6195-f2f4-485a-9970-388827a75cb7'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:17.736543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-863b6195-f2f4-485a-9970-388827a75cb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa826fecf0d44ad9f5a23c9c83c15ab', 'request_id': 'req-863b6195-f2f4-485a-9970-388827a75cb7'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:17.750252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-863b6195-f2f4-485a-9970-388827a75cb7 admin admin] The token cache is being invalidate because project bfa826fecf0d44ad9f5a23c9c83c15ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:36:17.751001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-863b6195-f2f4-485a-9970-388827a75cb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:17.752259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-863b6195-f2f4-485a-9970-388827a75cb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:17.758651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1212/2404] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:36:17 2022] DELETE /identity/v3/projects/bfa826fecf0d44ad9f5a23c9c83c15ab => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:17.761062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec34fb8-d26f-44a0-a4f1-6fea6f75e2d3 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:17.762417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec34fb8-d26f-44a0-a4f1-6fea6f75e2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:17.763148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec34fb8-d26f-44a0-a4f1-6fea6f75e2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:17.815749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-eec34fb8-d26f-44a0-a4f1-6fea6f75e2d3 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:17.818612 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-eec34fb8-d26f-44a0-a4f1-6fea6f75e2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:17.946544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1193/2405] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:17.954644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e551b5-25c3-4e06-b0ed-e69456518a8d None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:17.955397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e551b5-25c3-4e06-b0ed-e69456518a8d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:17.956136 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e551b5-25c3-4e06-b0ed-e69456518a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:17.982086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-10e551b5-25c3-4e06-b0ed-e69456518a8d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:17.983626 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-10e551b5-25c3-4e06-b0ed-e69456518a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:17.994916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1213/2406] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:18.003104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec34fb8-d26f-44a0-a4f1-6fea6f75e2d3 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.025381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec34fb8-d26f-44a0-a4f1-6fea6f75e2d3 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.032035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc385f3-aaf5-4a58-accb-868ca7cbe0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.032969 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc385f3-aaf5-4a58-accb-868ca7cbe0fb admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.033292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc385f3-aaf5-4a58-accb-868ca7cbe0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.033766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc385f3-aaf5-4a58-accb-868ca7cbe0fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.034641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc385f3-aaf5-4a58-accb-868ca7cbe0fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:18.034946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc385f3-aaf5-4a58-accb-868ca7cbe0fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.037686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc385f3-aaf5-4a58-accb-868ca7cbe0fb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.055929 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1194/2407] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:18.057231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e551b5-25c3-4e06-b0ed-e69456518a8d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.083241 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e551b5-25c3-4e06-b0ed-e69456518a8d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.098398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb1255-8e35-4b4c-ab65-cd468a909c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.098398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb1255-8e35-4b4c-ab65-cd468a909c98 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.098398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb1255-8e35-4b4c-ab65-cd468a909c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.098398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb1255-8e35-4b4c-ab65-cd468a909c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.098918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb1255-8e35-4b4c-ab65-cd468a909c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.098918 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb1255-8e35-4b4c-ab65-cd468a909c98 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.107578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:18.145935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1214/2408] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:36:18 2022] POST /identity/v3/projects => generated 381 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:18.150053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc385f3-aaf5-4a58-accb-868ca7cbe0fb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.172649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc385f3-aaf5-4a58-accb-868ca7cbe0fb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.288247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5c82a-5efe-425f-8b2f-3833cc799f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.289017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5c82a-5efe-425f-8b2f-3833cc799f74 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.289301 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5c82a-5efe-425f-8b2f-3833cc799f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.289594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5c82a-5efe-425f-8b2f-3833cc799f74 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.290295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5c82a-5efe-425f-8b2f-3833cc799f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.292494 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5c82a-5efe-425f-8b2f-3833cc799f74 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.325793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1195/2409] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:36:18 2022] POST /identity/v3/users => generated 291 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:18.330792 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb1255-8e35-4b4c-ab65-cd468a909c98 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.343961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb1255-8e35-4b4c-ab65-cd468a909c98 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.350154 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3821228-8807-467c-8a4f-bd23e9cc6803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.351147 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3821228-8807-467c-8a4f-bd23e9cc6803 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.351660 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3821228-8807-467c-8a4f-bd23e9cc6803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.352007 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3821228-8807-467c-8a4f-bd23e9cc6803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.353099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3821228-8807-467c-8a4f-bd23e9cc6803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.355451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3821228-8807-467c-8a4f-bd23e9cc6803 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.374760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1215/2410] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:36:18.379117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5c82a-5efe-425f-8b2f-3833cc799f74 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.397865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5c82a-5efe-425f-8b2f-3833cc799f74 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.408974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826095e-61d6-4673-80b5-ed607380749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.409765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826095e-61d6-4673-80b5-ed607380749f admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.410114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826095e-61d6-4673-80b5-ed607380749f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.410518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826095e-61d6-4673-80b5-ed607380749f admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.411277 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826095e-61d6-4673-80b5-ed607380749f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.413373 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826095e-61d6-4673-80b5-ed607380749f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.432579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1196/2411] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:36:18.433878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3821228-8807-467c-8a4f-bd23e9cc6803 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.451600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3821228-8807-467c-8a4f-bd23e9cc6803 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.457909 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b66c8f-943c-40e7-959d-479c0d921cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.460090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b66c8f-943c-40e7-959d-479c0d921cba admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.460605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b66c8f-943c-40e7-959d-479c0d921cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.461035 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b66c8f-943c-40e7-959d-479c0d921cba admin admin] PATH_INFO: `/v3/projects/b8abe8948c934248a448532711d4499b/users/b6a2a022db8b4ef98a8c4932f753240a/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.489691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b66c8f-943c-40e7-959d-479c0d921cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8abe8948c934248a448532711d4499b, user_id=b6a2a022db8b4ef98a8c4932f753240a, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.493345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b66c8f-943c-40e7-959d-479c0d921cba admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.515478 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1216/2412] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:36:18 2022] PUT /identity/v3/projects/b8abe8948c934248a448532711d4499b/users/b6a2a022db8b4ef98a8c4932f753240a/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:18.526078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40aced-c384-44e7-8427-716b6ac4e6e4 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.526408 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40aced-c384-44e7-8427-716b6ac4e6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.526644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40aced-c384-44e7-8427-716b6ac4e6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.573180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6d40aced-c384-44e7-8427-716b6ac4e6e4 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:18.575097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6d40aced-c384-44e7-8427-716b6ac4e6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:18.618472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:18.743103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1197/2413] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:18.756950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b66c8f-943c-40e7-959d-479c0d921cba admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.759319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-56b66c8f-943c-40e7-959d-479c0d921cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:18.776006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31d013-ffc4-41ba-89f8-ab44ff4d5e32 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.776967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31d013-ffc4-41ba-89f8-ab44ff4d5e32 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.776967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31d013-ffc4-41ba-89f8-ab44ff4d5e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.810490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5d31d013-ffc4-41ba-89f8-ab44ff4d5e32 None None] MFA Rules not processed for user `f93bd44f2fbc4be4be905c8b8ae0e343`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:18.811749 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5d31d013-ffc4-41ba-89f8-ab44ff4d5e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:18.905585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b66c8f-943c-40e7-959d-479c0d921cba admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.906234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1198/2414] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:36:18.913300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7f047-6aeb-42be-b21e-21808238487e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.914434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7f047-6aeb-42be-b21e-21808238487e service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.914960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7f047-6aeb-42be-b21e-21808238487e service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.915483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7f047-6aeb-42be-b21e-21808238487e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.934638 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7f047-6aeb-42be-b21e-21808238487e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.936722 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7f047-6aeb-42be-b21e-21808238487e service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.948529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1217/2415] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:20.340276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155dee9-361e-4d42-83c5-d50215db5931 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:20.341826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155dee9-361e-4d42-83c5-d50215db5931 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:20.342590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155dee9-361e-4d42-83c5-d50215db5931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:20.600077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8155dee9-361e-4d42-83c5-d50215db5931 None None] MFA Rules not processed for user `b6a2a022db8b4ef98a8c4932f753240a`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:20.608660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8155dee9-361e-4d42-83c5-d50215db5931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:20.817253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1199/2416] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:20.830433 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7f047-6aeb-42be-b21e-21808238487e service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:20.832860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0ae7f047-6aeb-42be-b21e-21808238487e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:21.032253 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7f047-6aeb-42be-b21e-21808238487e service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:21.046149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0ba97-c20c-41c7-afe6-b60a5fc223e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:21.049128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0ba97-c20c-41c7-afe6-b60a5fc223e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.049128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0ba97-c20c-41c7-afe6-b60a5fc223e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.050740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0ba97-c20c-41c7-afe6-b60a5fc223e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.079016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0ba97-c20c-41c7-afe6-b60a5fc223e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:21.082435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0ba97-c20c-41c7-afe6-b60a5fc223e6 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:21.111254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1218/2417] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:24.082337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f7319-6b39-4f87-ae08-1795715d2434 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:24.083204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f7319-6b39-4f87-ae08-1795715d2434 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:24.083204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f7319-6b39-4f87-ae08-1795715d2434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:24.120414 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3c8f7319-6b39-4f87-ae08-1795715d2434 None None] MFA Rules not processed for user `b6a2a022db8b4ef98a8c4932f753240a`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:24.134044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3c8f7319-6b39-4f87-ae08-1795715d2434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:24.151008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1200/2418] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:26.090289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0ba97-c20c-41c7-afe6-b60a5fc223e6 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:26.092132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-dcc0ba97-c20c-41c7-afe6-b60a5fc223e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:26.118333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0ba97-c20c-41c7-afe6-b60a5fc223e6 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:26.136166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:26.154700 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:26.156149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:26.159960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] PATH_INFO: `/v3/users/454c63976a6e4dd981e93d6912115c42` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:26.166486 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=454c63976a6e4dd981e93d6912115c42)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:26.180610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:26.428378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454c63976a6e4dd981e93d6912115c42'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:26.467492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:26.488898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454c63976a6e4dd981e93d6912115c42'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:26.491061 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454c63976a6e4dd981e93d6912115c42'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:26.534254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454c63976a6e4dd981e93d6912115c42'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:26.540271 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454c63976a6e4dd981e93d6912115c42'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:26.629661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1219/2419] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:36:26 2022] DELETE /identity/v3/users/454c63976a6e4dd981e93d6912115c42 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:26.635608 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e364ce-9af9-4f25-ac5f-b4175a6cf6ab None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:26.636695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e364ce-9af9-4f25-ac5f-b4175a6cf6ab None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:26.637134 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e364ce-9af9-4f25-ac5f-b4175a6cf6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:26.673518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-26e364ce-9af9-4f25-ac5f-b4175a6cf6ab None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:26.676765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-26e364ce-9af9-4f25-ac5f-b4175a6cf6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:26.892845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1201/2420] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:26.905459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:26.908025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:27.074593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1d5ea-5cc3-4e0a-b5fc-aecae461de2d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:27.085775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1844e1-e2e4-445f-a89e-fb5ca7d9a9a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:27.087511 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1844e1-e2e4-445f-a89e-fb5ca7d9a9a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:27.088245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1844e1-e2e4-445f-a89e-fb5ca7d9a9a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:27.088963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1844e1-e2e4-445f-a89e-fb5ca7d9a9a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:27.123316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1844e1-e2e4-445f-a89e-fb5ca7d9a9a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:27.125052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1844e1-e2e4-445f-a89e-fb5ca7d9a9a7 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:27.145090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1220/2421] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:27.908751 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e364ce-9af9-4f25-ac5f-b4175a6cf6ab None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:27.910904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-26e364ce-9af9-4f25-ac5f-b4175a6cf6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:27.924895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:27.932738 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e364ce-9af9-4f25-ac5f-b4175a6cf6ab None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:27.945795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d61a0f-44fb-4b08-8734-387036928148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:27.946690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d61a0f-44fb-4b08-8734-387036928148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:27.946978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d61a0f-44fb-4b08-8734-387036928148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:27.947253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d61a0f-44fb-4b08-8734-387036928148 admin admin] PATH_INFO: `/v3/projects/5f9272ab221d45aead1718872b27e545` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:27.948548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d61a0f-44fb-4b08-8734-387036928148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f9272ab221d45aead1718872b27e545)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:27.950477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d61a0f-44fb-4b08-8734-387036928148 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:28.041162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b9d61a0f-44fb-4b08-8734-387036928148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f9272ab221d45aead1718872b27e545', 'request_id': 'req-b9d61a0f-44fb-4b08-8734-387036928148'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:28.050131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b9d61a0f-44fb-4b08-8734-387036928148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f9272ab221d45aead1718872b27e545', 'request_id': 'req-b9d61a0f-44fb-4b08-8734-387036928148'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:28.072400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b9d61a0f-44fb-4b08-8734-387036928148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f9272ab221d45aead1718872b27e545', 'request_id': 'req-b9d61a0f-44fb-4b08-8734-387036928148'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:28.091632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b9d61a0f-44fb-4b08-8734-387036928148 admin admin] The token cache is being invalidate because project 5f9272ab221d45aead1718872b27e545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:36:28.092311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b9d61a0f-44fb-4b08-8734-387036928148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:28.096614 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b9d61a0f-44fb-4b08-8734-387036928148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:28.099255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1202/2422] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:36:27 2022] DELETE /identity/v3/projects/5f9272ab221d45aead1718872b27e545 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:28.104755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51984bf3-0e85-4a01-a2ab-c95b377bf23d None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.105191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51984bf3-0e85-4a01-a2ab-c95b377bf23d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.105582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51984bf3-0e85-4a01-a2ab-c95b377bf23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.125632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-51984bf3-0e85-4a01-a2ab-c95b377bf23d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:28.127781 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-51984bf3-0e85-4a01-a2ab-c95b377bf23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:28.216462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1221/2423] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:28.226385 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14378b9-49ce-4f1a-b670-4b3fc5cd5bc3 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.227082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14378b9-49ce-4f1a-b670-4b3fc5cd5bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.227615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14378b9-49ce-4f1a-b670-4b3fc5cd5bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.251646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c14378b9-49ce-4f1a-b670-4b3fc5cd5bc3 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:28.253655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c14378b9-49ce-4f1a-b670-4b3fc5cd5bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:28.271108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51984bf3-0e85-4a01-a2ab-c95b377bf23d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:28.273182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1203/2424] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:28.282519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51984bf3-0e85-4a01-a2ab-c95b377bf23d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:28.308260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe0250-401d-4970-9da4-c501a14ce95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:28.309076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe0250-401d-4970-9da4-c501a14ce95c admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.309370 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe0250-401d-4970-9da4-c501a14ce95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.309655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe0250-401d-4970-9da4-c501a14ce95c admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.310409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe0250-401d-4970-9da4-c501a14ce95c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:28.310903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe0250-401d-4970-9da4-c501a14ce95c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:28.312741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe0250-401d-4970-9da4-c501a14ce95c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:28.339397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14378b9-49ce-4f1a-b670-4b3fc5cd5bc3 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:28.350767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14378b9-49ce-4f1a-b670-4b3fc5cd5bc3 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:28.351552 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1222/2425] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:28.358280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957ad99-dc71-46b4-afe5-df0724c24eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:28.358978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957ad99-dc71-46b4-afe5-df0724c24eda admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.359280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957ad99-dc71-46b4-afe5-df0724c24eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.361138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957ad99-dc71-46b4-afe5-df0724c24eda admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.362327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957ad99-dc71-46b4-afe5-df0724c24eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:28.366723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957ad99-dc71-46b4-afe5-df0724c24eda admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:28.383991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe0250-401d-4970-9da4-c501a14ce95c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:28.386196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d8fe0250-401d-4970-9da4-c501a14ce95c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:28.403407 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe0250-401d-4970-9da4-c501a14ce95c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:28.412955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:28.414109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.414675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.415110 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] PATH_INFO: `/v3/users/b3d91f1bf80c4003976d7ebd84170c65` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.415594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1204/2426] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:36:28 2022] POST /identity/v3/projects => generated 369 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:36:28.416529 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957ad99-dc71-46b4-afe5-df0724c24eda admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:28.417249 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3d91f1bf80c4003976d7ebd84170c65)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:28.422841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:28.522642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3d91f1bf80c4003976d7ebd84170c65'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:28.529818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:28.540257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3d91f1bf80c4003976d7ebd84170c65'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:28.540992 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3d91f1bf80c4003976d7ebd84170c65'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:28.554441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957ad99-dc71-46b4-afe5-df0724c24eda admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:28.561389 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3d91f1bf80c4003976d7ebd84170c65'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:28.562259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3d91f1bf80c4003976d7ebd84170c65'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:28.569093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7643df-62b7-454e-9e29-5529248f8e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:28.570075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7643df-62b7-454e-9e29-5529248f8e4a admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.570375 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7643df-62b7-454e-9e29-5529248f8e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.570635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7643df-62b7-454e-9e29-5529248f8e4a admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.571337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7643df-62b7-454e-9e29-5529248f8e4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:28.575111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7643df-62b7-454e-9e29-5529248f8e4a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:28.615391 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1223/2427] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:36:28 2022] DELETE /identity/v3/users/b3d91f1bf80c4003976d7ebd84170c65 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:28.618794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1205/2428] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:36:28 2022] POST /identity/v3/users => generated 285 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:28.620413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:28.623502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:28.720182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70da56e-a846-4757-a505-5ecb49f2a4e3 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:28.725913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4afc0b-8f6a-4b77-9a53-bf25a69baa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:28.726960 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4afc0b-8f6a-4b77-9a53-bf25a69baa04 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.727089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4afc0b-8f6a-4b77-9a53-bf25a69baa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.727338 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4afc0b-8f6a-4b77-9a53-bf25a69baa04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.728517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4afc0b-8f6a-4b77-9a53-bf25a69baa04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:28.730701 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4afc0b-8f6a-4b77-9a53-bf25a69baa04 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:28.749864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1224/2429] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:36:28.751455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7643df-62b7-454e-9e29-5529248f8e4a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:28.769310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7643df-62b7-454e-9e29-5529248f8e4a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:28.775910 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d11bda-c8c3-44d5-9f83-9eea8c52520a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:28.776634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d11bda-c8c3-44d5-9f83-9eea8c52520a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.776920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d11bda-c8c3-44d5-9f83-9eea8c52520a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.777202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d11bda-c8c3-44d5-9f83-9eea8c52520a admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.778406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d11bda-c8c3-44d5-9f83-9eea8c52520a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:28.779616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d11bda-c8c3-44d5-9f83-9eea8c52520a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:28.800609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1206/2430] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:36:28.801664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4afc0b-8f6a-4b77-9a53-bf25a69baa04 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:28.822075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4afc0b-8f6a-4b77-9a53-bf25a69baa04 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:28.843285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0a4de-4241-4c95-8cd5-128a875690e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:28.844514 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0a4de-4241-4c95-8cd5-128a875690e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.844847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0a4de-4241-4c95-8cd5-128a875690e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.845128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0a4de-4241-4c95-8cd5-128a875690e9 admin admin] PATH_INFO: `/v3/projects/f024f50cbf814fba966c5aa8262885d4/users/40e1b93f53f7470392adda2aded3013b/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.881295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0a4de-4241-4c95-8cd5-128a875690e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f024f50cbf814fba966c5aa8262885d4, user_id=40e1b93f53f7470392adda2aded3013b, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:28.883985 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0a4de-4241-4c95-8cd5-128a875690e9 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:28.907569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1225/2431] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:36:28 2022] PUT /identity/v3/projects/f024f50cbf814fba966c5aa8262885d4/users/40e1b93f53f7470392adda2aded3013b/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:36:29.021263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89284c0d-aa20-4930-9cb7-7ca149afa318 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.021263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89284c0d-aa20-4930-9cb7-7ca149afa318 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.021263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89284c0d-aa20-4930-9cb7-7ca149afa318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.050933 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-89284c0d-aa20-4930-9cb7-7ca149afa318 None None] MFA Rules not processed for user `40e1b93f53f7470392adda2aded3013b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:29.052784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-89284c0d-aa20-4930-9cb7-7ca149afa318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:29.081421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:29.256631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0a4de-4241-4c95-8cd5-128a875690e9 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:29.257604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1207/2432] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:29.259950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0cb0a4de-4241-4c95-8cd5-128a875690e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:29.267426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89284c0d-aa20-4930-9cb7-7ca149afa318 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:29.268902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-89284c0d-aa20-4930-9cb7-7ca149afa318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:29.372904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89284c0d-aa20-4930-9cb7-7ca149afa318 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:29.379417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c90867-925b-4e3e-88db-0c4af5728eee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b67db080874f3ebb508239c7a8ccf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:29.379947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c90867-925b-4e3e-88db-0c4af5728eee service glance] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.380789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c90867-925b-4e3e-88db-0c4af5728eee service glance] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.381390 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c90867-925b-4e3e-88db-0c4af5728eee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.396086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c90867-925b-4e3e-88db-0c4af5728eee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:29.398577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c90867-925b-4e3e-88db-0c4af5728eee service glance] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:29.413131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1208/2433] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:36:29.479941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0a4de-4241-4c95-8cd5-128a875690e9 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:29.489382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:29.491493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.491943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.492370 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] PATH_INFO: `/v3/projects/c96d401f97834a2586165d4e7e0eef70` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.494050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c96d401f97834a2586165d4e7e0eef70)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:29.496076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:29.581488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:29.592493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c96d401f97834a2586165d4e7e0eef70', 'request_id': 'req-dff16b9c-8beb-4c5d-86b8-d240544e140f'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:29.606679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c96d401f97834a2586165d4e7e0eef70', 'request_id': 'req-dff16b9c-8beb-4c5d-86b8-d240544e140f'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:29.621940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c96d401f97834a2586165d4e7e0eef70', 'request_id': 'req-dff16b9c-8beb-4c5d-86b8-d240544e140f'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:29.638215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926717b-6fc2-44e6-a794-be5ae901042a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.638515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926717b-6fc2-44e6-a794-be5ae901042a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.639174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926717b-6fc2-44e6-a794-be5ae901042a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.657827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] The token cache is being invalidate because project c96d401f97834a2586165d4e7e0eef70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:36:29.658102 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:29.660891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:29.673143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1226/2434] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:36:29 2022] DELETE /identity/v3/projects/c96d401f97834a2586165d4e7e0eef70 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:29.678116 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:29.680277 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3926717b-6fc2-44e6-a794-be5ae901042a None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:29.681882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3926717b-6fc2-44e6-a794-be5ae901042a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:29.683285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:29.801666 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff16b9c-8beb-4c5d-86b8-d240544e140f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:29.808786 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:29.810563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.810968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.811230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] PATH_INFO: `/v3/users/56c55e99c31342c3af75a3a8f98cab54` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.812575 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56c55e99c31342c3af75a3a8f98cab54)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:29.812935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1209/2435] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 22:36:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:36:29.814727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:29.862519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926717b-6fc2-44e6-a794-be5ae901042a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:29.865049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3926717b-6fc2-44e6-a794-be5ae901042a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:29.921283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56c55e99c31342c3af75a3a8f98cab54'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:29.937693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56c55e99c31342c3af75a3a8f98cab54'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:29.938342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56c55e99c31342c3af75a3a8f98cab54'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:29.947155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56c55e99c31342c3af75a3a8f98cab54'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:29.947970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56c55e99c31342c3af75a3a8f98cab54'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:29.977453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1227/2436] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:36:29 2022] DELETE /identity/v3/users/56c55e99c31342c3af75a3a8f98cab54 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:30.022746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926717b-6fc2-44e6-a794-be5ae901042a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:30.044378 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:30.131662 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793feaf2-0dd2-4701-9b5c-9bd4772f8411 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:30.133663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793feaf2-0dd2-4701-9b5c-9bd4772f8411 service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.134309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793feaf2-0dd2-4701-9b5c-9bd4772f8411 service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.134690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793feaf2-0dd2-4701-9b5c-9bd4772f8411 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.137324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-793feaf2-0dd2-4701-9b5c-9bd4772f8411 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:30.289313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793feaf2-0dd2-4701-9b5c-9bd4772f8411 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:30.291251 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793feaf2-0dd2-4701-9b5c-9bd4772f8411 service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:30.304137 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1210/2437] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 22:36:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:36:30.360992 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:30.364132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:30.497519 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:30.535119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584d630-35f1-4e1f-9cac-1362fbf35921 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:30.544568 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e62603-c887-496c-a17c-9d07ea70460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:30.545451 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e62603-c887-496c-a17c-9d07ea70460f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.545737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e62603-c887-496c-a17c-9d07ea70460f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.546005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e62603-c887-496c-a17c-9d07ea70460f admin admin] PATH_INFO: `/v3/projects/fa22f60be7534926abf464eba1c17f5e` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.547240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e62603-c887-496c-a17c-9d07ea70460f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa22f60be7534926abf464eba1c17f5e)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:30.549230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e62603-c887-496c-a17c-9d07ea70460f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:30.627125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-31e62603-c887-496c-a17c-9d07ea70460f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa22f60be7534926abf464eba1c17f5e', 'request_id': 'req-31e62603-c887-496c-a17c-9d07ea70460f'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:30.648988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-31e62603-c887-496c-a17c-9d07ea70460f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa22f60be7534926abf464eba1c17f5e', 'request_id': 'req-31e62603-c887-496c-a17c-9d07ea70460f'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:30.657030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-31e62603-c887-496c-a17c-9d07ea70460f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa22f60be7534926abf464eba1c17f5e', 'request_id': 'req-31e62603-c887-496c-a17c-9d07ea70460f'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:30.682345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-31e62603-c887-496c-a17c-9d07ea70460f admin admin] The token cache is being invalidate because project fa22f60be7534926abf464eba1c17f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:36:30.682345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-31e62603-c887-496c-a17c-9d07ea70460f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:30.693771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-31e62603-c887-496c-a17c-9d07ea70460f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:30.697694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1228/2438] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:36:30 2022] DELETE /identity/v3/projects/fa22f60be7534926abf464eba1c17f5e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:30.704324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff62f7-7011-4d27-b882-c2cf4bc57063 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.705431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff62f7-7011-4d27-b882-c2cf4bc57063 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.705900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff62f7-7011-4d27-b882-c2cf4bc57063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.751160 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-93ff62f7-7011-4d27-b882-c2cf4bc57063 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:30.756947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-93ff62f7-7011-4d27-b882-c2cf4bc57063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:30.887628 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1211/2439] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:30.889473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822e1cb-07f7-4f42-b73b-69f518898b6a None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.894790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822e1cb-07f7-4f42-b73b-69f518898b6a None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.895082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822e1cb-07f7-4f42-b73b-69f518898b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.936530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-3822e1cb-07f7-4f42-b73b-69f518898b6a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:30.937816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3822e1cb-07f7-4f42-b73b-69f518898b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:30.952707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1229/2440] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:30.956752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff62f7-7011-4d27-b882-c2cf4bc57063 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:30.967707 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff62f7-7011-4d27-b882-c2cf4bc57063 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:30.975259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6544ea-5736-419a-84f5-f28e49e8adb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:30.975886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6544ea-5736-419a-84f5-f28e49e8adb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.976176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6544ea-5736-419a-84f5-f28e49e8adb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.976463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6544ea-5736-419a-84f5-f28e49e8adb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.977226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6544ea-5736-419a-84f5-f28e49e8adb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:30.977535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6544ea-5736-419a-84f5-f28e49e8adb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:30.979963 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6544ea-5736-419a-84f5-f28e49e8adb3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:31.001587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1212/2441] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:31.002962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822e1cb-07f7-4f42-b73b-69f518898b6a None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:31.019767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822e1cb-07f7-4f42-b73b-69f518898b6a None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:31.025058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddffc16-c81c-44b5-a74b-d4e32d7c54e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:31.025969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddffc16-c81c-44b5-a74b-d4e32d7c54e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:31.026365 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddffc16-c81c-44b5-a74b-d4e32d7c54e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:31.026567 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddffc16-c81c-44b5-a74b-d4e32d7c54e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:31.027285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddffc16-c81c-44b5-a74b-d4e32d7c54e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:31.029299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddffc16-c81c-44b5-a74b-d4e32d7c54e9 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:31.069307 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1230/2442] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:36:31 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:36:31.073717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6544ea-5736-419a-84f5-f28e49e8adb3 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:31.093463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6544ea-5736-419a-84f5-f28e49e8adb3 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:31.151153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:31.174122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f323c13-5de2-49a9-a0cc-0a469bf3a4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:31.174911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f323c13-5de2-49a9-a0cc-0a469bf3a4af admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:31.175297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f323c13-5de2-49a9-a0cc-0a469bf3a4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:31.175804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f323c13-5de2-49a9-a0cc-0a469bf3a4af admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:31.176728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f323c13-5de2-49a9-a0cc-0a469bf3a4af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:31.179886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f323c13-5de2-49a9-a0cc-0a469bf3a4af admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:31.210400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1213/2443] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:36:31 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:36:31.214845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddffc16-c81c-44b5-a74b-d4e32d7c54e9 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:31.230915 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddffc16-c81c-44b5-a74b-d4e32d7c54e9 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:31.243110 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633f895-9f5e-42ef-960b-a89c88bcde08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:31.246149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633f895-9f5e-42ef-960b-a89c88bcde08 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:31.246559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633f895-9f5e-42ef-960b-a89c88bcde08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:31.247055 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633f895-9f5e-42ef-960b-a89c88bcde08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:31.247940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633f895-9f5e-42ef-960b-a89c88bcde08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:31.250363 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633f895-9f5e-42ef-960b-a89c88bcde08 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:31.265003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1231/2444] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:36:31.270983 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f323c13-5de2-49a9-a0cc-0a469bf3a4af admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:31.281918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f323c13-5de2-49a9-a0cc-0a469bf3a4af admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:31.289518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eed1c-ef25-4da4-9c3c-ca210e7d8293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:31.290225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eed1c-ef25-4da4-9c3c-ca210e7d8293 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:31.290515 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eed1c-ef25-4da4-9c3c-ca210e7d8293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:31.290798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eed1c-ef25-4da4-9c3c-ca210e7d8293 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:31.291555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384eed1c-ef25-4da4-9c3c-ca210e7d8293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:31.293256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384eed1c-ef25-4da4-9c3c-ca210e7d8293 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:31.306097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1214/2445] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:36:31.310160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633f895-9f5e-42ef-960b-a89c88bcde08 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:31.325472 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633f895-9f5e-42ef-960b-a89c88bcde08 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:31.338482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139bb47-f6fb-416a-913f-3de299cb742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:31.339677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139bb47-f6fb-416a-913f-3de299cb742f admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:31.340040 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139bb47-f6fb-416a-913f-3de299cb742f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:31.340691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139bb47-f6fb-416a-913f-3de299cb742f admin admin] PATH_INFO: `/v3/projects/5ae7587a3a7a4cdd9e5a76748a5ca843/users/ef6e7950f469480cb6d3c38d382524dd/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:31.369159 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139bb47-f6fb-416a-913f-3de299cb742f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae7587a3a7a4cdd9e5a76748a5ca843, user_id=ef6e7950f469480cb6d3c38d382524dd, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:31.373646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139bb47-f6fb-416a-913f-3de299cb742f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:31.400985 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1232/2446] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:36:31 2022] PUT /identity/v3/projects/5ae7587a3a7a4cdd9e5a76748a5ca843/users/ef6e7950f469480cb6d3c38d382524dd/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:36:31.412137 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc955f-be05-4a92-9bea-cfa94e239c78 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:31.414120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc955f-be05-4a92-9bea-cfa94e239c78 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:31.414535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc955f-be05-4a92-9bea-cfa94e239c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:31.467536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-efdc955f-be05-4a92-9bea-cfa94e239c78 None None] MFA Rules not processed for user `ef6e7950f469480cb6d3c38d382524dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:31.469120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-efdc955f-be05-4a92-9bea-cfa94e239c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:31.592398 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1215/2447] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:31.609018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139bb47-f6fb-416a-913f-3de299cb742f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:31.611493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0139bb47-f6fb-416a-913f-3de299cb742f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:31.725115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139bb47-f6fb-416a-913f-3de299cb742f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:31.729704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b13608-217c-446d-aa96-1aae0eed6bdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:31.730374 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b13608-217c-446d-aa96-1aae0eed6bdb service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:31.730666 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b13608-217c-446d-aa96-1aae0eed6bdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:31.730948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b13608-217c-446d-aa96-1aae0eed6bdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:31.756777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b13608-217c-446d-aa96-1aae0eed6bdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:31.759423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b13608-217c-446d-aa96-1aae0eed6bdb service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:31.777810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1233/2448] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:36:31.805711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc955f-be05-4a92-9bea-cfa94e239c78 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:31.919569 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc955f-be05-4a92-9bea-cfa94e239c78 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:31.928275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:31.929517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:31.930029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:31.931355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] PATH_INFO: `/v3/users/ef6e7950f469480cb6d3c38d382524dd` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:31.933807 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6e7950f469480cb6d3c38d382524dd)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:31.937037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:32.048241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6e7950f469480cb6d3c38d382524dd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:32.055885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:32.059030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6e7950f469480cb6d3c38d382524dd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:32.075882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6e7950f469480cb6d3c38d382524dd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:32.085638 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6e7950f469480cb6d3c38d382524dd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:32.087144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6e7950f469480cb6d3c38d382524dd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:32.099309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1216/2449] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:36:31 2022] DELETE /identity/v3/users/ef6e7950f469480cb6d3c38d382524dd => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:32.101258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968415fd-154e-4587-a636-1b78876ca393 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:32.101693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968415fd-154e-4587-a636-1b78876ca393 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:32.102171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968415fd-154e-4587-a636-1b78876ca393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:32.125458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-968415fd-154e-4587-a636-1b78876ca393 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:32.126898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-968415fd-154e-4587-a636-1b78876ca393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:32.143021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:32.217039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1234/2450] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:32.225847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:32.228468 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:32.342621 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d4453-ebd5-40b5-9ab1-098e823e7a1b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:32.353060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4372f-acaf-440a-9633-f32fc75d6db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:32.355789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4372f-acaf-440a-9633-f32fc75d6db3 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:32.356125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4372f-acaf-440a-9633-f32fc75d6db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:32.356433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4372f-acaf-440a-9633-f32fc75d6db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:32.381655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4372f-acaf-440a-9633-f32fc75d6db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:32.383804 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4372f-acaf-440a-9633-f32fc75d6db3 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:32.399625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1217/2451] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:32.852930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968415fd-154e-4587-a636-1b78876ca393 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:32.854574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-968415fd-154e-4587-a636-1b78876ca393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:32.875002 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968415fd-154e-4587-a636-1b78876ca393 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:32.895192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c22010-819b-47d5-b54a-efe1dfac75e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:32.897757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c22010-819b-47d5-b54a-efe1dfac75e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:32.898610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c22010-819b-47d5-b54a-efe1dfac75e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:32.899107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c22010-819b-47d5-b54a-efe1dfac75e8 admin admin] PATH_INFO: `/v3/projects/5ae7587a3a7a4cdd9e5a76748a5ca843` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:32.902232 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c22010-819b-47d5-b54a-efe1dfac75e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ae7587a3a7a4cdd9e5a76748a5ca843)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:32.905184 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c22010-819b-47d5-b54a-efe1dfac75e8 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:33.040004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d8c22010-819b-47d5-b54a-efe1dfac75e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ae7587a3a7a4cdd9e5a76748a5ca843', 'request_id': 'req-d8c22010-819b-47d5-b54a-efe1dfac75e8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:33.065197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d8c22010-819b-47d5-b54a-efe1dfac75e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ae7587a3a7a4cdd9e5a76748a5ca843', 'request_id': 'req-d8c22010-819b-47d5-b54a-efe1dfac75e8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:33.077953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d8c22010-819b-47d5-b54a-efe1dfac75e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ae7587a3a7a4cdd9e5a76748a5ca843', 'request_id': 'req-d8c22010-819b-47d5-b54a-efe1dfac75e8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:33.099667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d8c22010-819b-47d5-b54a-efe1dfac75e8 admin admin] The token cache is being invalidate because project 5ae7587a3a7a4cdd9e5a76748a5ca843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:36:33.100288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d8c22010-819b-47d5-b54a-efe1dfac75e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:33.101370 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-d8c22010-819b-47d5-b54a-efe1dfac75e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:33.109214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1235/2452] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:36:32 2022] DELETE /identity/v3/projects/5ae7587a3a7a4cdd9e5a76748a5ca843 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:33.109885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc164c-77c6-4b6e-ab2b-200d5bf4d018 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:33.110339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc164c-77c6-4b6e-ab2b-200d5bf4d018 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:33.110422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc164c-77c6-4b6e-ab2b-200d5bf4d018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:33.141536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9cdc164c-77c6-4b6e-ab2b-200d5bf4d018 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:33.142782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9cdc164c-77c6-4b6e-ab2b-200d5bf4d018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:33.338641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08923cd2-7df8-4d2d-8062-5d322482d81e None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:33.338955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08923cd2-7df8-4d2d-8062-5d322482d81e None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:33.339318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08923cd2-7df8-4d2d-8062-5d322482d81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:33.350346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1218/2453] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:36:33.369832 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-08923cd2-7df8-4d2d-8062-5d322482d81e None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:33.371546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-08923cd2-7df8-4d2d-8062-5d322482d81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:33.400610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc164c-77c6-4b6e-ab2b-200d5bf4d018 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:33.410743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1236/2454] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:33.416232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc164c-77c6-4b6e-ab2b-200d5bf4d018 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:33.420875 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef69ead-6350-4c70-bd6a-cf0150c81ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:33.421690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef69ead-6350-4c70-bd6a-cf0150c81ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:33.422177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef69ead-6350-4c70-bd6a-cf0150c81ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:33.422569 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef69ead-6350-4c70-bd6a-cf0150c81ef8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:33.423404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef69ead-6350-4c70-bd6a-cf0150c81ef8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:33.423831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef69ead-6350-4c70-bd6a-cf0150c81ef8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:33.425724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef69ead-6350-4c70-bd6a-cf0150c81ef8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:33.449144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08923cd2-7df8-4d2d-8062-5d322482d81e None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:33.461412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1219/2455] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:33.461725 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08923cd2-7df8-4d2d-8062-5d322482d81e None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:33.467543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388c57d-d048-48d0-8813-1fae5f47de29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:33.468388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388c57d-d048-48d0-8813-1fae5f47de29 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:33.468787 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388c57d-d048-48d0-8813-1fae5f47de29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:33.469072 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388c57d-d048-48d0-8813-1fae5f47de29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:33.469934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388c57d-d048-48d0-8813-1fae5f47de29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:33.472147 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388c57d-d048-48d0-8813-1fae5f47de29 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:33.505652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1237/2456] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:36:33 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:33.510560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef69ead-6350-4c70-bd6a-cf0150c81ef8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:33.524133 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef69ead-6350-4c70-bd6a-cf0150c81ef8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:33.617471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991dc165-9bbb-4a07-b0aa-249f851b28d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:33.618825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991dc165-9bbb-4a07-b0aa-249f851b28d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:33.618825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991dc165-9bbb-4a07-b0aa-249f851b28d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:33.619514 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991dc165-9bbb-4a07-b0aa-249f851b28d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:33.620311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991dc165-9bbb-4a07-b0aa-249f851b28d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:33.623209 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991dc165-9bbb-4a07-b0aa-249f851b28d3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:33.657801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:33.705901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1220/2457] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:36:33 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:33.710032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388c57d-d048-48d0-8813-1fae5f47de29 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:33.735910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388c57d-d048-48d0-8813-1fae5f47de29 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:33.757583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a46d82-7089-4a08-9aa2-7bcb019cb16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:33.759408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a46d82-7089-4a08-9aa2-7bcb019cb16e admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:33.759767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a46d82-7089-4a08-9aa2-7bcb019cb16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:33.760139 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a46d82-7089-4a08-9aa2-7bcb019cb16e admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:33.760987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a46d82-7089-4a08-9aa2-7bcb019cb16e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:33.762972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a46d82-7089-4a08-9aa2-7bcb019cb16e admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:33.780097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1238/2458] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:36:33.784136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991dc165-9bbb-4a07-b0aa-249f851b28d3 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:33.801297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991dc165-9bbb-4a07-b0aa-249f851b28d3 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:33.811287 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd4a6b-3480-407f-9c0a-a591c56f0924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:33.812811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd4a6b-3480-407f-9c0a-a591c56f0924 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:33.813336 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd4a6b-3480-407f-9c0a-a591c56f0924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:33.813954 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd4a6b-3480-407f-9c0a-a591c56f0924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:33.815105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd4a6b-3480-407f-9c0a-a591c56f0924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:33.817365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd4a6b-3480-407f-9c0a-a591c56f0924 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:33.838049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1221/2459] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:36:33.842743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a46d82-7089-4a08-9aa2-7bcb019cb16e admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:33.874614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a46d82-7089-4a08-9aa2-7bcb019cb16e admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:33.885436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cb331-3c1b-48a6-90ef-d0469b51952c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:33.886947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cb331-3c1b-48a6-90ef-d0469b51952c admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:33.887238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cb331-3c1b-48a6-90ef-d0469b51952c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:33.887579 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cb331-3c1b-48a6-90ef-d0469b51952c admin admin] PATH_INFO: `/v3/projects/f7d6ddf1c9874b03958b0c935219d567/users/2f01c370c86e4f0fab0c9d2246e6e1d0/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:33.915420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cb331-3c1b-48a6-90ef-d0469b51952c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d6ddf1c9874b03958b0c935219d567, user_id=2f01c370c86e4f0fab0c9d2246e6e1d0, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:33.926199 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cb331-3c1b-48a6-90ef-d0469b51952c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:33.979269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1239/2460] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:36:33 2022] PUT /identity/v3/projects/f7d6ddf1c9874b03958b0c935219d567/users/2f01c370c86e4f0fab0c9d2246e6e1d0/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:34.053328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70224b-d33c-40c0-82ee-8bc5ec467326 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:34.053714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70224b-d33c-40c0-82ee-8bc5ec467326 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:34.053896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70224b-d33c-40c0-82ee-8bc5ec467326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:34.145682 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9b70224b-d33c-40c0-82ee-8bc5ec467326 None None] MFA Rules not processed for user `2f01c370c86e4f0fab0c9d2246e6e1d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:34.147114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9b70224b-d33c-40c0-82ee-8bc5ec467326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:34.259457 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1222/2461] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:34.272854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cb331-3c1b-48a6-90ef-d0469b51952c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:34.274919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f72cb331-3c1b-48a6-90ef-d0469b51952c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:34.334654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:34.357386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cb331-3c1b-48a6-90ef-d0469b51952c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:34.362742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba404812-57d8-4fee-8f0e-bb813015ca36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:34.363497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404812-57d8-4fee-8f0e-bb813015ca36 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:34.363883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404812-57d8-4fee-8f0e-bb813015ca36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:34.365565 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba404812-57d8-4fee-8f0e-bb813015ca36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:34.376195 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba404812-57d8-4fee-8f0e-bb813015ca36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:34.377771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba404812-57d8-4fee-8f0e-bb813015ca36 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:34.390559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1240/2462] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:42.256411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445789d7-2fe2-4263-bdb0-37541a00ba4b None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:42.257365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445789d7-2fe2-4263-bdb0-37541a00ba4b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:42.257365 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445789d7-2fe2-4263-bdb0-37541a00ba4b None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:42.259003 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1223/2463] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 22:36:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:36:42.272362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ead45b-da1d-429c-bc13-7fe3cb7477ce None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:42.273020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ead45b-da1d-429c-bc13-7fe3cb7477ce None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:42.273254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ead45b-da1d-429c-bc13-7fe3cb7477ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:42.314280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-d1ead45b-da1d-429c-bc13-7fe3cb7477ce None None] MFA Rules not processed for user `c77f6ccc7cb84c04ac532007a04a0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:42.315664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d1ead45b-da1d-429c-bc13-7fe3cb7477ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:42.500394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1241/2464] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 22:36:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:43.936362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445789d7-2fe2-4263-bdb0-37541a00ba4b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:44.025093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445789d7-2fe2-4263-bdb0-37541a00ba4b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:44.030506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:44.031416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:44.032213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:44.032705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] PATH_INFO: `/v3/users/2f01c370c86e4f0fab0c9d2246e6e1d0` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:44.034256 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f01c370c86e4f0fab0c9d2246e6e1d0)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:44.036588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:44.107189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f01c370c86e4f0fab0c9d2246e6e1d0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:44.119708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f01c370c86e4f0fab0c9d2246e6e1d0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:44.134089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:44.135714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f01c370c86e4f0fab0c9d2246e6e1d0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:44.159898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f01c370c86e4f0fab0c9d2246e6e1d0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:44.160695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f01c370c86e4f0fab0c9d2246e6e1d0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:44.179469 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1224/2465] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:36:43 2022] DELETE /identity/v3/users/2f01c370c86e4f0fab0c9d2246e6e1d0 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:44.181438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae82ddc-4235-4943-9c25-8f8dc4e16611 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:44.181900 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae82ddc-4235-4943-9c25-8f8dc4e16611 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:44.182148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae82ddc-4235-4943-9c25-8f8dc4e16611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:44.216793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-2ae82ddc-4235-4943-9c25-8f8dc4e16611 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:44.219065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2ae82ddc-4235-4943-9c25-8f8dc4e16611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:44.315718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1242/2466] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:44.326589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:44.328453 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:44.446882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e9d8a-c861-4823-b8dc-9fa7bfb77d47 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:44.450949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5b7df-adcd-49c4-bc34-e4c9444824c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:44.451526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5b7df-adcd-49c4-bc34-e4c9444824c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:44.451826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5b7df-adcd-49c4-bc34-e4c9444824c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:44.452354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5b7df-adcd-49c4-bc34-e4c9444824c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:44.463224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5b7df-adcd-49c4-bc34-e4c9444824c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:44.464833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5b7df-adcd-49c4-bc34-e4c9444824c8 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:44.491518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1225/2467] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:45.197543 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae82ddc-4235-4943-9c25-8f8dc4e16611 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:45.199074 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2ae82ddc-4235-4943-9c25-8f8dc4e16611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:45.237184 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae82ddc-4235-4943-9c25-8f8dc4e16611 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:45.243672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23015412-ceb0-4106-913c-f6b424cbb5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:45.244608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23015412-ceb0-4106-913c-f6b424cbb5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:45.244900 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23015412-ceb0-4106-913c-f6b424cbb5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:45.245177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23015412-ceb0-4106-913c-f6b424cbb5ec admin admin] PATH_INFO: `/v3/projects/f7d6ddf1c9874b03958b0c935219d567` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:45.246655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23015412-ceb0-4106-913c-f6b424cbb5ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d6ddf1c9874b03958b0c935219d567)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:45.248856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23015412-ceb0-4106-913c-f6b424cbb5ec admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:45.321535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-23015412-ceb0-4106-913c-f6b424cbb5ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d6ddf1c9874b03958b0c935219d567', 'request_id': 'req-23015412-ceb0-4106-913c-f6b424cbb5ec'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:45.337665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-23015412-ceb0-4106-913c-f6b424cbb5ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d6ddf1c9874b03958b0c935219d567', 'request_id': 'req-23015412-ceb0-4106-913c-f6b424cbb5ec'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:45.345353 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-23015412-ceb0-4106-913c-f6b424cbb5ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d6ddf1c9874b03958b0c935219d567', 'request_id': 'req-23015412-ceb0-4106-913c-f6b424cbb5ec'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:45.377332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:45.391342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-23015412-ceb0-4106-913c-f6b424cbb5ec admin admin] The token cache is being invalidate because project f7d6ddf1c9874b03958b0c935219d567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:36:45.391654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-23015412-ceb0-4106-913c-f6b424cbb5ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:45.394313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-23015412-ceb0-4106-913c-f6b424cbb5ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:36:45.412806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1243/2468] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:36:45 2022] DELETE /identity/v3/projects/f7d6ddf1c9874b03958b0c935219d567 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:48.150449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5b7df-adcd-49c4-bc34-e4c9444824c8 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:48.152217 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e8c5b7df-adcd-49c4-bc34-e4c9444824c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:48.231955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5b7df-adcd-49c4-bc34-e4c9444824c8 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:48.295879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9dce3-c2b7-47d4-a1b1-437506619c94 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b67db080874f3ebb508239c7a8ccf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:48.296503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9dce3-c2b7-47d4-a1b1-437506619c94 service glance] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:48.296789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9dce3-c2b7-47d4-a1b1-437506619c94 service glance] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:48.297059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9dce3-c2b7-47d4-a1b1-437506619c94 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:48.299217 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7fa9dce3-c2b7-47d4-a1b1-437506619c94 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:48.364421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:48.382727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9dce3-c2b7-47d4-a1b1-437506619c94 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:48.384331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9dce3-c2b7-47d4-a1b1-437506619c94 service glance] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:48.396398 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1226/2469] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:48.531188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde039ea-8108-4bf7-aa84-3846cec6c693 None None] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:48.531686 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde039ea-8108-4bf7-aa84-3846cec6c693 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:48.531771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde039ea-8108-4bf7-aa84-3846cec6c693 None None] PATH_INFO: `/` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:48.533154 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1244/2470] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 22:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:48.538929 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04fc79-846e-4f8e-9234-129e6fadf546 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:48.539288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04fc79-846e-4f8e-9234-129e6fadf546 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:48.539654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04fc79-846e-4f8e-9234-129e6fadf546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:48.571450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4c04fc79-846e-4f8e-9234-129e6fadf546 None None] MFA Rules not processed for user `8a120d9e7e204dc5b972fd9402897b41`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:48.573742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4c04fc79-846e-4f8e-9234-129e6fadf546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:48.671689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1227/2471] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 22:36:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:48.684687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde039ea-8108-4bf7-aa84-3846cec6c693 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:48.686506 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-cde039ea-8108-4bf7-aa84-3846cec6c693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:48.709654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde039ea-8108-4bf7-aa84-3846cec6c693 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:48.716252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5dce7b-1796-480a-a7af-492b2af243b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:48.717084 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5dce7b-1796-480a-a7af-492b2af243b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:48.717379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5dce7b-1796-480a-a7af-492b2af243b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:48.718204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5dce7b-1796-480a-a7af-492b2af243b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:48.732422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5dce7b-1796-480a-a7af-492b2af243b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:48.735303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5dce7b-1796-480a-a7af-492b2af243b0 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:48.745868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1245/2472] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:48.937417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd512cdc-93d5-4798-917e-b0191d14d0df None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:48.937859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd512cdc-93d5-4798-917e-b0191d14d0df None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:48.938258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd512cdc-93d5-4798-917e-b0191d14d0df None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:48.940149 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1228/2473] 10.222.0.42 () {54 vars in 921 bytes} [Thu Jul 21 22:36:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:48.955677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7756e-2ce6-4184-9fdb-a0b58499df6f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:48.956011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7756e-2ce6-4184-9fdb-a0b58499df6f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:48.956579 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7756e-2ce6-4184-9fdb-a0b58499df6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:49.007886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-fca7756e-2ce6-4184-9fdb-a0b58499df6f None None] MFA Rules not processed for user `c77f6ccc7cb84c04ac532007a04a0f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:49.009264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fca7756e-2ce6-4184-9fdb-a0b58499df6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:49.022661 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1246/2474] 10.222.0.42 () {58 vars in 1018 bytes} [Thu Jul 21 22:36:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:36:49.033676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd512cdc-93d5-4798-917e-b0191d14d0df None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:49.037286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bd512cdc-93d5-4798-917e-b0191d14d0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:49.057507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd512cdc-93d5-4798-917e-b0191d14d0df None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:49.064106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa4f5e-bb15-4e2f-9da3-ea2aaf791374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:49.065051 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa4f5e-bb15-4e2f-9da3-ea2aaf791374 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:49.065408 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa4f5e-bb15-4e2f-9da3-ea2aaf791374 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:49.065702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa4f5e-bb15-4e2f-9da3-ea2aaf791374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:49.080276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa4f5e-bb15-4e2f-9da3-ea2aaf791374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:49.082554 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa4f5e-bb15-4e2f-9da3-ea2aaf791374 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:49.096883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1229/2475] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:36:50.729670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7756e-2ce6-4184-9fdb-a0b58499df6f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:50.731758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fca7756e-2ce6-4184-9fdb-a0b58499df6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:50.810492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7756e-2ce6-4184-9fdb-a0b58499df6f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:50.815664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f4939-e489-4944-931e-1af6b0c6432c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:50.817272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f4939-e489-4944-931e-1af6b0c6432c service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:50.817580 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f4939-e489-4944-931e-1af6b0c6432c service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:50.817883 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f4939-e489-4944-931e-1af6b0c6432c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:50.819755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f94f4939-e489-4944-931e-1af6b0c6432c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:50.835748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f4939-e489-4944-931e-1af6b0c6432c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:50.837362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f4939-e489-4944-931e-1af6b0c6432c service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:50.848953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1247/2476] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:36:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:36:56.868328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316b525-4bc7-49f9-b29e-c80134e28cc6 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:56.868879 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316b525-4bc7-49f9-b29e-c80134e28cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:56.868977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316b525-4bc7-49f9-b29e-c80134e28cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:56.887757 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5316b525-4bc7-49f9-b29e-c80134e28cc6 None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:56.888980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5316b525-4bc7-49f9-b29e-c80134e28cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:56.947131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1230/2477] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 22:36:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:36:56.959069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f4939-e489-4944-931e-1af6b0c6432c service placement] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:56.960571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f94f4939-e489-4944-931e-1af6b0c6432c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:57.031764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f4939-e489-4944-931e-1af6b0c6432c service placement] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:57.037762 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558e5ca-18ad-4460-8506-4ec57eb29917 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:57.038425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558e5ca-18ad-4460-8506-4ec57eb29917 service swift] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:57.038762 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558e5ca-18ad-4460-8506-4ec57eb29917 service swift] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:57.039028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558e5ca-18ad-4460-8506-4ec57eb29917 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:57.047659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558e5ca-18ad-4460-8506-4ec57eb29917 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:57.049293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558e5ca-18ad-4460-8506-4ec57eb29917 service swift] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:57.058241 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1248/2478] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 22:36:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:37:04.961905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb513ae7-3a77-43db-8c82-de3545dfa943 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:04.962868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb513ae7-3a77-43db-8c82-de3545dfa943 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:04.963317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb513ae7-3a77-43db-8c82-de3545dfa943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:04.988959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-eb513ae7-3a77-43db-8c82-de3545dfa943 None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:04.990930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-eb513ae7-3a77-43db-8c82-de3545dfa943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:05.000513 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1231/2479] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jul 21 22:37:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:05.014346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558e5ca-18ad-4460-8506-4ec57eb29917 service swift] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:05.026375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558e5ca-18ad-4460-8506-4ec57eb29917 service swift] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:05.032248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1f5a5-f476-4bb0-a83e-bd39e2b6185d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:05.033195 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1f5a5-f476-4bb0-a83e-bd39e2b6185d service swift] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:05.033547 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1f5a5-f476-4bb0-a83e-bd39e2b6185d service swift] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:05.033975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1f5a5-f476-4bb0-a83e-bd39e2b6185d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:05.044312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1f5a5-f476-4bb0-a83e-bd39e2b6185d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:05.046200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1f5a5-f476-4bb0-a83e-bd39e2b6185d service swift] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:05.055059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1249/2480] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 22:37:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:37:06.275221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb513ae7-3a77-43db-8c82-de3545dfa943 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:06.276713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-eb513ae7-3a77-43db-8c82-de3545dfa943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:06.352170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb513ae7-3a77-43db-8c82-de3545dfa943 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:06.356544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:06.357511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:06.357798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:06.358076 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] PATH_INFO: `/v3/users/40e1b93f53f7470392adda2aded3013b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:06.359380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e1b93f53f7470392adda2aded3013b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:06.361576 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:06.391686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:06.414943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e1b93f53f7470392adda2aded3013b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:06.422330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e1b93f53f7470392adda2aded3013b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:06.439338 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e1b93f53f7470392adda2aded3013b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:06.449967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e1b93f53f7470392adda2aded3013b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:06.451006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e1b93f53f7470392adda2aded3013b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:06.460624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1232/2481] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:37:06 2022] DELETE /identity/v3/users/40e1b93f53f7470392adda2aded3013b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:37:06.465169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278132ae-ee8d-466d-8f64-e9e3d69fa646 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:06.465807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278132ae-ee8d-466d-8f64-e9e3d69fa646 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:06.465807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278132ae-ee8d-466d-8f64-e9e3d69fa646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:06.490152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-278132ae-ee8d-466d-8f64-e9e3d69fa646 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:06.492053 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-278132ae-ee8d-466d-8f64-e9e3d69fa646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:06.549369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:06.630168 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1250/2482] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:06.649622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:06.654711 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:06.809074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b4361-d4b7-4e0e-b15a-d36f17f9d620 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:06.824957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138689e2-c039-4310-876a-c493c16d3210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:06.825987 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138689e2-c039-4310-876a-c493c16d3210 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:06.826269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138689e2-c039-4310-876a-c493c16d3210 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:06.826571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138689e2-c039-4310-876a-c493c16d3210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:06.841034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138689e2-c039-4310-876a-c493c16d3210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:06.842511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138689e2-c039-4310-876a-c493c16d3210 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:06.858372 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1233/2483] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:37:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:37:07.766108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278132ae-ee8d-466d-8f64-e9e3d69fa646 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:07.767608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-278132ae-ee8d-466d-8f64-e9e3d69fa646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:07.782282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278132ae-ee8d-466d-8f64-e9e3d69fa646 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:07.788207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b14b09-2e5e-4b21-a162-5934b6b9502c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:07.789030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b14b09-2e5e-4b21-a162-5934b6b9502c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:07.789322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b14b09-2e5e-4b21-a162-5934b6b9502c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:07.789605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b14b09-2e5e-4b21-a162-5934b6b9502c admin admin] PATH_INFO: `/v3/projects/f024f50cbf814fba966c5aa8262885d4` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:07.790983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b14b09-2e5e-4b21-a162-5934b6b9502c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f024f50cbf814fba966c5aa8262885d4)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:07.792961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b14b09-2e5e-4b21-a162-5934b6b9502c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:07.817905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138689e2-c039-4310-876a-c493c16d3210 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:07.818790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-138689e2-c039-4310-876a-c493c16d3210 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:07.833454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138689e2-c039-4310-876a-c493c16d3210 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:07.838795 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e824ec-8804-437f-ad8d-48c59a9a5613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:07.839674 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e824ec-8804-437f-ad8d-48c59a9a5613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:07.839958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e824ec-8804-437f-ad8d-48c59a9a5613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:07.840403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e824ec-8804-437f-ad8d-48c59a9a5613 admin admin] PATH_INFO: `/v3/users/35f252f0338547aea73d1eb7c81ca199` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:07.841633 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e824ec-8804-437f-ad8d-48c59a9a5613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f252f0338547aea73d1eb7c81ca199)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:07.843563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e824ec-8804-437f-ad8d-48c59a9a5613 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:07.853961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-90b14b09-2e5e-4b21-a162-5934b6b9502c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f024f50cbf814fba966c5aa8262885d4', 'request_id': 'req-90b14b09-2e5e-4b21-a162-5934b6b9502c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:07.861505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-90b14b09-2e5e-4b21-a162-5934b6b9502c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f024f50cbf814fba966c5aa8262885d4', 'request_id': 'req-90b14b09-2e5e-4b21-a162-5934b6b9502c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:07.867814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-90b14b09-2e5e-4b21-a162-5934b6b9502c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f024f50cbf814fba966c5aa8262885d4', 'request_id': 'req-90b14b09-2e5e-4b21-a162-5934b6b9502c'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:07.881533 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-90b14b09-2e5e-4b21-a162-5934b6b9502c admin admin] The token cache is being invalidate because project f024f50cbf814fba966c5aa8262885d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:37:07.881826 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-90b14b09-2e5e-4b21-a162-5934b6b9502c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:07.882702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-90b14b09-2e5e-4b21-a162-5934b6b9502c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:07.884361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1251/2484] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:37:07 2022] DELETE /identity/v3/projects/f024f50cbf814fba966c5aa8262885d4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:37:07.889518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06962ce8-b338-4b9b-9174-7c2516db38a8 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:07.889817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06962ce8-b338-4b9b-9174-7c2516db38a8 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:07.890100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06962ce8-b338-4b9b-9174-7c2516db38a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:07.894104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-44e824ec-8804-437f-ad8d-48c59a9a5613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f252f0338547aea73d1eb7c81ca199'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:07.901883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-44e824ec-8804-437f-ad8d-48c59a9a5613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f252f0338547aea73d1eb7c81ca199'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:07.912257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-06962ce8-b338-4b9b-9174-7c2516db38a8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:07.913644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-06962ce8-b338-4b9b-9174-7c2516db38a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:07.915224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-44e824ec-8804-437f-ad8d-48c59a9a5613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f252f0338547aea73d1eb7c81ca199'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:07.922610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-44e824ec-8804-437f-ad8d-48c59a9a5613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f252f0338547aea73d1eb7c81ca199'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:07.923486 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-44e824ec-8804-437f-ad8d-48c59a9a5613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f252f0338547aea73d1eb7c81ca199'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:07.929928 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1234/2485] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:37:07 2022] DELETE /identity/v3/users/35f252f0338547aea73d1eb7c81ca199 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:37:07.978087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1252/2486] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:37:07.986017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78c742-fb1b-4890-9913-cb828eb01df8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:07.986316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78c742-fb1b-4890-9913-cb828eb01df8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:07.986599 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78c742-fb1b-4890-9913-cb828eb01df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.004622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4e78c742-fb1b-4890-9913-cb828eb01df8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:08.005964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4e78c742-fb1b-4890-9913-cb828eb01df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:08.018551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:08.076693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1235/2487] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:08.081033 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06962ce8-b338-4b9b-9174-7c2516db38a8 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.095975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06962ce8-b338-4b9b-9174-7c2516db38a8 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.103152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe29d9-d907-4c20-b7d3-58a5b4c4079b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.104194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe29d9-d907-4c20-b7d3-58a5b4c4079b admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.104622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe29d9-d907-4c20-b7d3-58a5b4c4079b admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.105109 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe29d9-d907-4c20-b7d3-58a5b4c4079b admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.106738 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe29d9-d907-4c20-b7d3-58a5b4c4079b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:37:08.107253 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe29d9-d907-4c20-b7d3-58a5b4c4079b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.110122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe29d9-d907-4c20-b7d3-58a5b4c4079b admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.121279 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78c742-fb1b-4890-9913-cb828eb01df8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.123498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4e78c742-fb1b-4890-9913-cb828eb01df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:08.141917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78c742-fb1b-4890-9913-cb828eb01df8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.142325 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1253/2488] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:08.148668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe29d9-d907-4c20-b7d3-58a5b4c4079b admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.155929 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.157754 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.158253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.158860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0 admin admin] PATH_INFO: `/v3/projects/58636dfe0f8c463bab0031b5d6734bd9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.161620 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58636dfe0f8c463bab0031b5d6734bd9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.164517 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.169868 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe29d9-d907-4c20-b7d3-58a5b4c4079b admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.177476 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75f916-7009-44b2-874b-b717fba38c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.178632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75f916-7009-44b2-874b-b717fba38c40 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.179178 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75f916-7009-44b2-874b-b717fba38c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.179614 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75f916-7009-44b2-874b-b717fba38c40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.180907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75f916-7009-44b2-874b-b717fba38c40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.184348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75f916-7009-44b2-874b-b717fba38c40 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.220989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1254/2489] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:37:08 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:37:08.222151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75f916-7009-44b2-874b-b717fba38c40 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.227069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58636dfe0f8c463bab0031b5d6734bd9', 'request_id': 'req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:08.235600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58636dfe0f8c463bab0031b5d6734bd9', 'request_id': 'req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:08.244389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58636dfe0f8c463bab0031b5d6734bd9', 'request_id': 'req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:08.258458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0 admin admin] The token cache is being invalidate because project 58636dfe0f8c463bab0031b5d6734bd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:37:08.258859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:08.259676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2d592b39-f6b7-4a4e-be7a-f5239b5e45a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:37:08.264073 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:08.265763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1236/2490] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:37:08 2022] DELETE /identity/v3/projects/58636dfe0f8c463bab0031b5d6734bd9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:37:08.267296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836714a8-d2fd-4425-968c-238d2eeadc2d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.267588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836714a8-d2fd-4425-968c-238d2eeadc2d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.267873 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836714a8-d2fd-4425-968c-238d2eeadc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.289501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-836714a8-d2fd-4425-968c-238d2eeadc2d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:08.290755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-836714a8-d2fd-4425-968c-238d2eeadc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:08.295726 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75f916-7009-44b2-874b-b717fba38c40 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.298798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1237/2491] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:08.300655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9c2c3-ea9b-408e-ab6e-a2149e83934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.301533 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9c2c3-ea9b-408e-ab6e-a2149e83934d admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.301757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9c2c3-ea9b-408e-ab6e-a2149e83934d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.302031 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9c2c3-ea9b-408e-ab6e-a2149e83934d admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.302746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9c2c3-ea9b-408e-ab6e-a2149e83934d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.304669 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9c2c3-ea9b-408e-ab6e-a2149e83934d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.306473 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a1a0a-c0cc-4119-a923-ba2d508ef073 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.306767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a1a0a-c0cc-4119-a923-ba2d508ef073 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.307053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a1a0a-c0cc-4119-a923-ba2d508ef073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.329588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1255/2492] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:37:08 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:37:08.329816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a04a1a0a-c0cc-4119-a923-ba2d508ef073 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:08.329816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a04a1a0a-c0cc-4119-a923-ba2d508ef073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:08.331144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9c2c3-ea9b-408e-ab6e-a2149e83934d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.333410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-69c9c2c3-ea9b-408e-ab6e-a2149e83934d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:08.336588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1238/2493] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:37:08.340542 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a1a0a-c0cc-4119-a923-ba2d508ef073 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.346219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9c2c3-ea9b-408e-ab6e-a2149e83934d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.349832 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a1a0a-c0cc-4119-a923-ba2d508ef073 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.353725 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad31e98-cd68-4db4-831c-9fd9388b6874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.354399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad31e98-cd68-4db4-831c-9fd9388b6874 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.354525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e9fcd-0d15-45c8-a5d9-d78d44e7d47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.354679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad31e98-cd68-4db4-831c-9fd9388b6874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.354955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad31e98-cd68-4db4-831c-9fd9388b6874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.355073 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e9fcd-0d15-45c8-a5d9-d78d44e7d47f admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.355362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e9fcd-0d15-45c8-a5d9-d78d44e7d47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.355649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e9fcd-0d15-45c8-a5d9-d78d44e7d47f admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.355757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad31e98-cd68-4db4-831c-9fd9388b6874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.356376 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e9fcd-0d15-45c8-a5d9-d78d44e7d47f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:37:08.356719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e9fcd-0d15-45c8-a5d9-d78d44e7d47f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.357322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad31e98-cd68-4db4-831c-9fd9388b6874 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.358311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e9fcd-0d15-45c8-a5d9-d78d44e7d47f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.368553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1256/2494] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:37:08.370305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1239/2495] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:37:08.373069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad31e98-cd68-4db4-831c-9fd9388b6874 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.374249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e9fcd-0d15-45c8-a5d9-d78d44e7d47f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.384627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad31e98-cd68-4db4-831c-9fd9388b6874 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.384901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e9fcd-0d15-45c8-a5d9-d78d44e7d47f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.389183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e4407-3739-444d-897d-ce3f8da605f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.389397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480727c2-218c-4905-94d5-479ad37f5456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.389842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e4407-3739-444d-897d-ce3f8da605f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.390039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480727c2-218c-4905-94d5-479ad37f5456 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.390150 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e4407-3739-444d-897d-ce3f8da605f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.390393 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e4407-3739-444d-897d-ce3f8da605f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.390595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480727c2-218c-4905-94d5-479ad37f5456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.390880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480727c2-218c-4905-94d5-479ad37f5456 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.391124 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e4407-3739-444d-897d-ce3f8da605f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.391713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480727c2-218c-4905-94d5-479ad37f5456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.392772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e4407-3739-444d-897d-ce3f8da605f3 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.393642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480727c2-218c-4905-94d5-479ad37f5456 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.404190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1257/2496] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:37:08.409038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e4407-3739-444d-897d-ce3f8da605f3 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.420157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e4407-3739-444d-897d-ce3f8da605f3 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.425410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ad3bf-d2bd-40c8-9ce8-677e273f7f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.426556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ad3bf-d2bd-40c8-9ce8-677e273f7f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.426845 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ad3bf-d2bd-40c8-9ce8-677e273f7f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.427131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ad3bf-d2bd-40c8-9ce8-677e273f7f38 admin admin] PATH_INFO: `/v3/projects/671dbc43a5c04a26b23e278f3f7964bb/users/68f8c3b4d761425db457093edfccce6b/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.440522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1240/2497] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:37:08 2022] POST /identity/v3/projects => generated 363 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:37:08.445694 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ad3bf-d2bd-40c8-9ce8-677e273f7f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=671dbc43a5c04a26b23e278f3f7964bb, user_id=68f8c3b4d761425db457093edfccce6b, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.445856 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480727c2-218c-4905-94d5-479ad37f5456 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.448746 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ad3bf-d2bd-40c8-9ce8-677e273f7f38 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.465754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1258/2498] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:37:08 2022] PUT /identity/v3/projects/671dbc43a5c04a26b23e278f3f7964bb/users/68f8c3b4d761425db457093edfccce6b/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:37:08.470596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c69de-2e91-4c7d-9e1c-d94cb52ea85f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.470911 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c69de-2e91-4c7d-9e1c-d94cb52ea85f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.471205 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c69de-2e91-4c7d-9e1c-d94cb52ea85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.488556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-6c9c69de-2e91-4c7d-9e1c-d94cb52ea85f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:08.489968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6c9c69de-2e91-4c7d-9e1c-d94cb52ea85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:08.513056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:08.530560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480727c2-218c-4905-94d5-479ad37f5456 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.567023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1259/2499] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:37:08.576372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c69de-2e91-4c7d-9e1c-d94cb52ea85f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.578282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6c9c69de-2e91-4c7d-9e1c-d94cb52ea85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:08.595587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18705fe-d4e9-4862-820a-f0b37ab5cc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.596352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18705fe-d4e9-4862-820a-f0b37ab5cc02 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.596739 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18705fe-d4e9-4862-820a-f0b37ab5cc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.597120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18705fe-d4e9-4862-820a-f0b37ab5cc02 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.597982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18705fe-d4e9-4862-820a-f0b37ab5cc02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.599998 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18705fe-d4e9-4862-820a-f0b37ab5cc02 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.623689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1241/2500] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:37:08 2022] POST /identity/v3/users => generated 282 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:37:08.627898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18705fe-d4e9-4862-820a-f0b37ab5cc02 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.638362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18705fe-d4e9-4862-820a-f0b37ab5cc02 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.639329 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c69de-2e91-4c7d-9e1c-d94cb52ea85f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.643003 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31c460-2f19-4744-ab28-bed6982c6885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.643641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93162155-b418-458c-b32b-70f510b74fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.643775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31c460-2f19-4744-ab28-bed6982c6885 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.643956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31c460-2f19-4744-ab28-bed6982c6885 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.644233 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31c460-2f19-4744-ab28-bed6982c6885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.644337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93162155-b418-458c-b32b-70f510b74fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.644835 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93162155-b418-458c-b32b-70f510b74fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.645314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93162155-b418-458c-b32b-70f510b74fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.646046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93162155-b418-458c-b32b-70f510b74fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.647786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93162155-b418-458c-b32b-70f510b74fdb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.656468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31c460-2f19-4744-ab28-bed6982c6885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.657935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31c460-2f19-4744-ab28-bed6982c6885 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.659170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1242/2501] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:37:08.663240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93162155-b418-458c-b32b-70f510b74fdb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.669896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1260/2502] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:37:08.672902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93162155-b418-458c-b32b-70f510b74fdb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.678166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc449e-4525-4825-82ff-75d9cb748a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.678823 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc449e-4525-4825-82ff-75d9cb748a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.679101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc449e-4525-4825-82ff-75d9cb748a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.679377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc449e-4525-4825-82ff-75d9cb748a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.680097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc449e-4525-4825-82ff-75d9cb748a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.681735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc449e-4525-4825-82ff-75d9cb748a03 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.694863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1243/2503] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:37:08.699222 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31c460-2f19-4744-ab28-bed6982c6885 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.709433 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31c460-2f19-4744-ab28-bed6982c6885 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.714161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0625e5b-ff3d-4167-84a5-b225fea9cbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.715054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0625e5b-ff3d-4167-84a5-b225fea9cbc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.715335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0625e5b-ff3d-4167-84a5-b225fea9cbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.715609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0625e5b-ff3d-4167-84a5-b225fea9cbc1 admin admin] PATH_INFO: `/v3/projects/fa53ac0d0979464fab80339ce82bc8f6/users/6c332efbe4dc4223b5854aead97a6677/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.731988 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0625e5b-ff3d-4167-84a5-b225fea9cbc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa53ac0d0979464fab80339ce82bc8f6, user_id=6c332efbe4dc4223b5854aead97a6677, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.735797 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0625e5b-ff3d-4167-84a5-b225fea9cbc1 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.740923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:08.756186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1261/2504] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:37:08 2022] PUT /identity/v3/projects/fa53ac0d0979464fab80339ce82bc8f6/users/6c332efbe4dc4223b5854aead97a6677/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:37:08.769812 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61183946-ee91-4e27-bc8c-f753ec934f3f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.770501 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61183946-ee91-4e27-bc8c-f753ec934f3f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.770915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61183946-ee91-4e27-bc8c-f753ec934f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.801722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-61183946-ee91-4e27-bc8c-f753ec934f3f None None] MFA Rules not processed for user `6c332efbe4dc4223b5854aead97a6677`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:08.803356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-61183946-ee91-4e27-bc8c-f753ec934f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:08.888053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1244/2505] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:37:08 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:08.899588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0625e5b-ff3d-4167-84a5-b225fea9cbc1 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:08.901359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c0625e5b-ff3d-4167-84a5-b225fea9cbc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:08.970857 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0625e5b-ff3d-4167-84a5-b225fea9cbc1 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:08.975282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c6d79-8068-4ff1-8736-d7c9c057b991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:08.975989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c6d79-8068-4ff1-8736-d7c9c057b991 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:08.976272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c6d79-8068-4ff1-8736-d7c9c057b991 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:08.976578 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c6d79-8068-4ff1-8736-d7c9c057b991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:08.985765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c6d79-8068-4ff1-8736-d7c9c057b991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:08.987466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c6d79-8068-4ff1-8736-d7c9c057b991 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:08.998283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1262/2506] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:37:08 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:37:13.052129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365b19-334d-41e0-a9d2-022a7afeee2b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:13.053551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365b19-334d-41e0-a9d2-022a7afeee2b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:13.054421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365b19-334d-41e0-a9d2-022a7afeee2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:13.095440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f1365b19-334d-41e0-a9d2-022a7afeee2b None None] MFA Rules not processed for user `68f8c3b4d761425db457093edfccce6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:13.097878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f1365b19-334d-41e0-a9d2-022a7afeee2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:13.167840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:13.190957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1245/2507] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:37:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:13.195704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c6d79-8068-4ff1-8736-d7c9c057b991 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:13.275296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c6d79-8068-4ff1-8736-d7c9c057b991 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:13.280011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e5a11-60ec-44a2-90b5-1bcf4a407174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:13.280743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e5a11-60ec-44a2-90b5-1bcf4a407174 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:13.281028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e5a11-60ec-44a2-90b5-1bcf4a407174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:13.281303 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e5a11-60ec-44a2-90b5-1bcf4a407174 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:13.282022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e5a11-60ec-44a2-90b5-1bcf4a407174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:13.283948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e5a11-60ec-44a2-90b5-1bcf4a407174 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:13.317594 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1263/2508] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:37:13 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:37:13.319084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1365b19-334d-41e0-a9d2-022a7afeee2b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:13.332437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1365b19-334d-41e0-a9d2-022a7afeee2b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:13.429138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc066594-b6bc-46bb-99bd-89eed579d0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:13.429966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc066594-b6bc-46bb-99bd-89eed579d0ac admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:13.430258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc066594-b6bc-46bb-99bd-89eed579d0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:13.430537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc066594-b6bc-46bb-99bd-89eed579d0ac admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:13.431232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc066594-b6bc-46bb-99bd-89eed579d0ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:13.433417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc066594-b6bc-46bb-99bd-89eed579d0ac admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:13.461062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1246/2509] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:37:13 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:37:13.464955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e5a11-60ec-44a2-90b5-1bcf4a407174 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:13.476197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e5a11-60ec-44a2-90b5-1bcf4a407174 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:13.481651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c5ee5-8807-41c1-ab9c-f249f4cdc8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:13.482419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c5ee5-8807-41c1-ab9c-f249f4cdc8a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:13.482712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c5ee5-8807-41c1-ab9c-f249f4cdc8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:13.483016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c5ee5-8807-41c1-ab9c-f249f4cdc8a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:13.483747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c5ee5-8807-41c1-ab9c-f249f4cdc8a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:13.486106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c5ee5-8807-41c1-ab9c-f249f4cdc8a2 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:13.504585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1264/2510] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:37:13.506377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc066594-b6bc-46bb-99bd-89eed579d0ac admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:13.523412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc066594-b6bc-46bb-99bd-89eed579d0ac admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:13.528701 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7511a3-652d-45b1-b37b-78cbcb0baf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:13.529687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7511a3-652d-45b1-b37b-78cbcb0baf04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:13.530036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7511a3-652d-45b1-b37b-78cbcb0baf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:13.530563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7511a3-652d-45b1-b37b-78cbcb0baf04 admin admin] PATH_INFO: `/v3/projects/617c3f22038947e3a09375406705efdb/users/7f396e9269b046db8d9eec2e86ecd534/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:13.549615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7511a3-652d-45b1-b37b-78cbcb0baf04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=617c3f22038947e3a09375406705efdb, user_id=7f396e9269b046db8d9eec2e86ecd534, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:13.552864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7511a3-652d-45b1-b37b-78cbcb0baf04 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:13.577894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1247/2511] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:37:13 2022] PUT /identity/v3/projects/617c3f22038947e3a09375406705efdb/users/7f396e9269b046db8d9eec2e86ecd534/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:37:13.584216 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c5ee5-8807-41c1-ab9c-f249f4cdc8a2 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:13.600278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c5ee5-8807-41c1-ab9c-f249f4cdc8a2 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:13.674569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d5c9f-5990-45f5-b847-b19029dc2f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:13.675295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d5c9f-5990-45f5-b847-b19029dc2f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:13.675646 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d5c9f-5990-45f5-b847-b19029dc2f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:13.675939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d5c9f-5990-45f5-b847-b19029dc2f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:13.676724 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d5c9f-5990-45f5-b847-b19029dc2f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:13.678785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d5c9f-5990-45f5-b847-b19029dc2f71 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:13.692010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1265/2512] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:37:13.695575 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7511a3-652d-45b1-b37b-78cbcb0baf04 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:13.707864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7511a3-652d-45b1-b37b-78cbcb0baf04 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:13.713269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b872831-aa49-4dda-9b51-4ab6a30753ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:13.714195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b872831-aa49-4dda-9b51-4ab6a30753ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:13.715166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b872831-aa49-4dda-9b51-4ab6a30753ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:13.715166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b872831-aa49-4dda-9b51-4ab6a30753ef admin admin] PATH_INFO: `/v3/projects/617c3f22038947e3a09375406705efdb/users/7f396e9269b046db8d9eec2e86ecd534/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:13.717215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b872831-aa49-4dda-9b51-4ab6a30753ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=617c3f22038947e3a09375406705efdb, user_id=7f396e9269b046db8d9eec2e86ecd534, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:13.720344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b872831-aa49-4dda-9b51-4ab6a30753ef admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:13.734748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1248/2513] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:37:13 2022] PUT /identity/v3/projects/617c3f22038947e3a09375406705efdb/users/7f396e9269b046db8d9eec2e86ecd534/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:37:20.054461 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cda147-70e9-4094-bf3b-4368662857b0 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:20.055397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cda147-70e9-4094-bf3b-4368662857b0 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:20.055575 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cda147-70e9-4094-bf3b-4368662857b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:20.086692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:20.115258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-34cda147-70e9-4094-bf3b-4368662857b0 None None] MFA Rules not processed for user `7f396e9269b046db8d9eec2e86ecd534`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:20.117592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-34cda147-70e9-4094-bf3b-4368662857b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:20.208209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1266/2514] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:37:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:20.231976 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b872831-aa49-4dda-9b51-4ab6a30753ef admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:20.234066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0b872831-aa49-4dda-9b51-4ab6a30753ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:20.364755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b872831-aa49-4dda-9b51-4ab6a30753ef admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:20.375133 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620987c3-5df9-47c9-8840-a097852a1b88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:20.376598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620987c3-5df9-47c9-8840-a097852a1b88 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:20.377107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620987c3-5df9-47c9-8840-a097852a1b88 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:20.377642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620987c3-5df9-47c9-8840-a097852a1b88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:20.442491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:20.474224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620987c3-5df9-47c9-8840-a097852a1b88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:20.476106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620987c3-5df9-47c9-8840-a097852a1b88 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:20.488530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1249/2515] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:37:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:37:22.025121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f465b9-3ac7-41dd-a4ec-89ae8711ec58 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:22.025622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f465b9-3ac7-41dd-a4ec-89ae8711ec58 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:22.025715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f465b9-3ac7-41dd-a4ec-89ae8711ec58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:22.058024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-37f465b9-3ac7-41dd-a4ec-89ae8711ec58 None None] MFA Rules not processed for user `68f8c3b4d761425db457093edfccce6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:22.059653 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-37f465b9-3ac7-41dd-a4ec-89ae8711ec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:22.068191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1267/2516] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:37:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:26.457266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b88ec-11c4-4168-8fa7-61922dd574d6 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:26.457946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b88ec-11c4-4168-8fa7-61922dd574d6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:26.458093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b88ec-11c4-4168-8fa7-61922dd574d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:26.499259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-770b88ec-11c4-4168-8fa7-61922dd574d6 None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:26.501000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-770b88ec-11c4-4168-8fa7-61922dd574d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:26.610946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1250/2517] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jul 21 22:37:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:37:26.655923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f465b9-3ac7-41dd-a4ec-89ae8711ec58 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:26.665369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-37f465b9-3ac7-41dd-a4ec-89ae8711ec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:26.804540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f465b9-3ac7-41dd-a4ec-89ae8711ec58 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:26.821028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4e42b-4071-4bdd-a8c7-f26c22a6470e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:26.822029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4e42b-4071-4bdd-a8c7-f26c22a6470e service swift] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:26.822518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4e42b-4071-4bdd-a8c7-f26c22a6470e service swift] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:26.822954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4e42b-4071-4bdd-a8c7-f26c22a6470e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:26.840529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4e42b-4071-4bdd-a8c7-f26c22a6470e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:26.842798 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4e42b-4071-4bdd-a8c7-f26c22a6470e service swift] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:26.856212 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1268/2518] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 22:37:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:37:36.596392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=36 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:36.851783 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=65 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:55.171915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de921089-83f1-4ba5-a227-525f05d7ac1c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:55.172576 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de921089-83f1-4ba5-a227-525f05d7ac1c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:55.172576 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de921089-83f1-4ba5-a227-525f05d7ac1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:55.191180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-de921089-83f1-4ba5-a227-525f05d7ac1c None None] MFA Rules not processed for user `b6a2a022db8b4ef98a8c4932f753240a`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:55.193235 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-de921089-83f1-4ba5-a227-525f05d7ac1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:55.264877 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1251/2519] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:37:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:05.250544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=25 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:05.260091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4e42b-4071-4bdd-a8c7-f26c22a6470e service swift] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:05.344300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4e42b-4071-4bdd-a8c7-f26c22a6470e service swift] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:05.353371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:05.354672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:05.355169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:05.355670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] PATH_INFO: `/v3/users/6c332efbe4dc4223b5854aead97a6677` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:05.357863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c332efbe4dc4223b5854aead97a6677)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:05.361761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:05.453209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c332efbe4dc4223b5854aead97a6677'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.463124 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c332efbe4dc4223b5854aead97a6677'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.464036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c332efbe4dc4223b5854aead97a6677'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.473309 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c332efbe4dc4223b5854aead97a6677'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.474076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c332efbe4dc4223b5854aead97a6677'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:05.493835 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1269/2520] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:05 2022] DELETE /identity/v3/users/6c332efbe4dc4223b5854aead97a6677 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:05.505341 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493495a4-37af-4ae7-97f0-9dee1b1bf0fa None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:05.505626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493495a4-37af-4ae7-97f0-9dee1b1bf0fa None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:05.505912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493495a4-37af-4ae7-97f0-9dee1b1bf0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:05.526186 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-493495a4-37af-4ae7-97f0-9dee1b1bf0fa None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:05.527548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-493495a4-37af-4ae7-97f0-9dee1b1bf0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:05.611664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1252/2521] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:38:05.624735 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:05.630181 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:05.718358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95affa21-5c54-4a02-af8f-2d7cdb181857 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:05.724161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74131d6c-448a-4d25-a860-94f15d53a8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:05.725365 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74131d6c-448a-4d25-a860-94f15d53a8af service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:05.725869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74131d6c-448a-4d25-a860-94f15d53a8af service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:05.726376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74131d6c-448a-4d25-a860-94f15d53a8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:05.748641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74131d6c-448a-4d25-a860-94f15d53a8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:05.751475 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74131d6c-448a-4d25-a860-94f15d53a8af service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:05.773567 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1270/2522] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:06.140718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493495a4-37af-4ae7-97f0-9dee1b1bf0fa None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:06.142394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-493495a4-37af-4ae7-97f0-9dee1b1bf0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:06.158593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493495a4-37af-4ae7-97f0-9dee1b1bf0fa None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:06.165048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218a511-63b9-4439-9d72-fddabd9ca4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:06.165943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218a511-63b9-4439-9d72-fddabd9ca4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:06.166228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218a511-63b9-4439-9d72-fddabd9ca4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:06.166500 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218a511-63b9-4439-9d72-fddabd9ca4d9 admin admin] PATH_INFO: `/v3/projects/fa53ac0d0979464fab80339ce82bc8f6` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:06.167781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218a511-63b9-4439-9d72-fddabd9ca4d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa53ac0d0979464fab80339ce82bc8f6)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:06.169913 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218a511-63b9-4439-9d72-fddabd9ca4d9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:06.232102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4218a511-63b9-4439-9d72-fddabd9ca4d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa53ac0d0979464fab80339ce82bc8f6', 'request_id': 'req-4218a511-63b9-4439-9d72-fddabd9ca4d9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:06.240215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4218a511-63b9-4439-9d72-fddabd9ca4d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa53ac0d0979464fab80339ce82bc8f6', 'request_id': 'req-4218a511-63b9-4439-9d72-fddabd9ca4d9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:06.249508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4218a511-63b9-4439-9d72-fddabd9ca4d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa53ac0d0979464fab80339ce82bc8f6', 'request_id': 'req-4218a511-63b9-4439-9d72-fddabd9ca4d9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:06.264263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4218a511-63b9-4439-9d72-fddabd9ca4d9 admin admin] The token cache is being invalidate because project fa53ac0d0979464fab80339ce82bc8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:06.264999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4218a511-63b9-4439-9d72-fddabd9ca4d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:06.265403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4218a511-63b9-4439-9d72-fddabd9ca4d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:06.267475 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1253/2523] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:06 2022] DELETE /identity/v3/projects/fa53ac0d0979464fab80339ce82bc8f6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:38:06.274403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40955bdf-aeb7-44b8-8bc9-31af00c978a0 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:06.275345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40955bdf-aeb7-44b8-8bc9-31af00c978a0 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:06.275630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40955bdf-aeb7-44b8-8bc9-31af00c978a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:06.295767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-40955bdf-aeb7-44b8-8bc9-31af00c978a0 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:06.297054 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-40955bdf-aeb7-44b8-8bc9-31af00c978a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:06.318368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:06.386466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1271/2524] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:06.395064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614df4bd-986c-4e55-a40e-9dfaa3bc8d28 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:06.395543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614df4bd-986c-4e55-a40e-9dfaa3bc8d28 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:06.395894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614df4bd-986c-4e55-a40e-9dfaa3bc8d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:06.423006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-614df4bd-986c-4e55-a40e-9dfaa3bc8d28 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:06.424346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-614df4bd-986c-4e55-a40e-9dfaa3bc8d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:06.433840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1254/2525] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:06.438192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40955bdf-aeb7-44b8-8bc9-31af00c978a0 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:06.449649 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40955bdf-aeb7-44b8-8bc9-31af00c978a0 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:06.454496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d214e-6bff-432d-bb27-07984feebe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:06.455192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d214e-6bff-432d-bb27-07984feebe34 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:06.455479 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d214e-6bff-432d-bb27-07984feebe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:06.455759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d214e-6bff-432d-bb27-07984feebe34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:06.457683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d214e-6bff-432d-bb27-07984feebe34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:06.458006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d214e-6bff-432d-bb27-07984feebe34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:06.459672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d214e-6bff-432d-bb27-07984feebe34 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:06.478323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1272/2526] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:06.482823 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614df4bd-986c-4e55-a40e-9dfaa3bc8d28 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:06.492923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614df4bd-986c-4e55-a40e-9dfaa3bc8d28 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:06.497585 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411eae5-74f3-4fb0-9885-5d1d3803c8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:06.498266 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411eae5-74f3-4fb0-9885-5d1d3803c8b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:06.498544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411eae5-74f3-4fb0-9885-5d1d3803c8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:06.498818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411eae5-74f3-4fb0-9885-5d1d3803c8b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:06.499527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411eae5-74f3-4fb0-9885-5d1d3803c8b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:06.501481 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411eae5-74f3-4fb0-9885-5d1d3803c8b3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:06.531420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1255/2527] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:06 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:06.538495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d214e-6bff-432d-bb27-07984feebe34 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:06.549880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d214e-6bff-432d-bb27-07984feebe34 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:06.609470 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62264781-dc03-423d-9d12-3262b29ffc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:06.610153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62264781-dc03-423d-9d12-3262b29ffc32 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:06.610432 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62264781-dc03-423d-9d12-3262b29ffc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:06.610706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62264781-dc03-423d-9d12-3262b29ffc32 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:06.611403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62264781-dc03-423d-9d12-3262b29ffc32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:06.613436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62264781-dc03-423d-9d12-3262b29ffc32 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:06.635502 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1273/2528] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:06 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:06.639650 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411eae5-74f3-4fb0-9885-5d1d3803c8b3 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:06.651196 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411eae5-74f3-4fb0-9885-5d1d3803c8b3 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:06.655951 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbc869-7e5c-4222-9cb9-c5f0fcdd1616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:06.656665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbc869-7e5c-4222-9cb9-c5f0fcdd1616 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:06.656951 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbc869-7e5c-4222-9cb9-c5f0fcdd1616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:06.657232 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbc869-7e5c-4222-9cb9-c5f0fcdd1616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:06.657965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbc869-7e5c-4222-9cb9-c5f0fcdd1616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:06.659584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbc869-7e5c-4222-9cb9-c5f0fcdd1616 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:06.670571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1256/2529] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:06.674865 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62264781-dc03-423d-9d12-3262b29ffc32 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:06.686637 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62264781-dc03-423d-9d12-3262b29ffc32 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:06.692067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da66bf0-1f37-4183-b486-e388f3dd7efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:06.692765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da66bf0-1f37-4183-b486-e388f3dd7efb admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:06.693133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da66bf0-1f37-4183-b486-e388f3dd7efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:06.693323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da66bf0-1f37-4183-b486-e388f3dd7efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:06.694052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da66bf0-1f37-4183-b486-e388f3dd7efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:06.695641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da66bf0-1f37-4183-b486-e388f3dd7efb admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:06.707635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1274/2530] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:06.711934 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbc869-7e5c-4222-9cb9-c5f0fcdd1616 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:06.723079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbc869-7e5c-4222-9cb9-c5f0fcdd1616 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:06.727830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ceb7f-c85a-4e62-86f5-3748dd67b4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:06.728771 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ceb7f-c85a-4e62-86f5-3748dd67b4ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:06.729146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ceb7f-c85a-4e62-86f5-3748dd67b4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:06.729527 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ceb7f-c85a-4e62-86f5-3748dd67b4ac admin admin] PATH_INFO: `/v3/projects/44c01c6c01ff43b99bdd5f17cfc6d723/users/42fe110e5cd34eb6889127ceb135b436/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:06.746838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ceb7f-c85a-4e62-86f5-3748dd67b4ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c01c6c01ff43b99bdd5f17cfc6d723, user_id=42fe110e5cd34eb6889127ceb135b436, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:06.749893 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ceb7f-c85a-4e62-86f5-3748dd67b4ac admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:06.771285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1257/2531] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:06 2022] PUT /identity/v3/projects/44c01c6c01ff43b99bdd5f17cfc6d723/users/42fe110e5cd34eb6889127ceb135b436/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:06.782956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0964114-2cb2-4a78-a97a-cd6ab15ffbee None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:06.783251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0964114-2cb2-4a78-a97a-cd6ab15ffbee None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:06.783535 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0964114-2cb2-4a78-a97a-cd6ab15ffbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:06.822671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-b0964114-2cb2-4a78-a97a-cd6ab15ffbee None None] MFA Rules not processed for user `42fe110e5cd34eb6889127ceb135b436`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:06.824023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b0964114-2cb2-4a78-a97a-cd6ab15ffbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:06.900182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1275/2532] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:38:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:06.912246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ceb7f-c85a-4e62-86f5-3748dd67b4ac admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:06.914312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fa1ceb7f-c85a-4e62-86f5-3748dd67b4ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:06.949507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:07.007741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ceb7f-c85a-4e62-86f5-3748dd67b4ac admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:07.012845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88ebb0-2ae6-46c3-a4c6-baac57a00765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:07.013746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88ebb0-2ae6-46c3-a4c6-baac57a00765 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:07.014113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88ebb0-2ae6-46c3-a4c6-baac57a00765 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:07.014463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88ebb0-2ae6-46c3-a4c6-baac57a00765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:07.030023 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88ebb0-2ae6-46c3-a4c6-baac57a00765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:07.031764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88ebb0-2ae6-46c3-a4c6-baac57a00765 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:07.042595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1258/2533] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:07.071017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0964114-2cb2-4a78-a97a-cd6ab15ffbee None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:07.100508 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:07.163226 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0964114-2cb2-4a78-a97a-cd6ab15ffbee None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:07.168700 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:07.169793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:07.170320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:07.170709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] PATH_INFO: `/v3/users/42fe110e5cd34eb6889127ceb135b436` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:07.172637 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=42fe110e5cd34eb6889127ceb135b436)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:07.174274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:07.239201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42fe110e5cd34eb6889127ceb135b436'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:07.247428 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42fe110e5cd34eb6889127ceb135b436'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:07.248201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42fe110e5cd34eb6889127ceb135b436'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:07.256268 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42fe110e5cd34eb6889127ceb135b436'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:07.257131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42fe110e5cd34eb6889127ceb135b436'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:07.277183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1276/2534] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:38:07 2022] DELETE /identity/v3/users/42fe110e5cd34eb6889127ceb135b436 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:07.281939 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e04929-63e5-4387-ae52-50b6cd63700d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:07.282391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e04929-63e5-4387-ae52-50b6cd63700d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:07.282500 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e04929-63e5-4387-ae52-50b6cd63700d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:07.301724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f5e04929-63e5-4387-ae52-50b6cd63700d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:07.302948 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f5e04929-63e5-4387-ae52-50b6cd63700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:07.365921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1259/2535] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:38:07.375719 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:07.377189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:07.437480 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a12c30-8cc6-476a-a199-2fa5523eab1c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:07.441825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2d5d2-5abe-4bab-b7f7-8fc3fa290bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:07.442609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2d5d2-5abe-4bab-b7f7-8fc3fa290bae service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:07.442913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2d5d2-5abe-4bab-b7f7-8fc3fa290bae service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:07.443179 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2d5d2-5abe-4bab-b7f7-8fc3fa290bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:07.454188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2d5d2-5abe-4bab-b7f7-8fc3fa290bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:07.455673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2d5d2-5abe-4bab-b7f7-8fc3fa290bae service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:07.468740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1277/2536] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:07.873786 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e04929-63e5-4387-ae52-50b6cd63700d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:07.875761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f5e04929-63e5-4387-ae52-50b6cd63700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:07.898245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e04929-63e5-4387-ae52-50b6cd63700d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:07.904233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bac3c-5581-4e26-9f94-e3f126b7af54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:07.905534 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bac3c-5581-4e26-9f94-e3f126b7af54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:07.905915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bac3c-5581-4e26-9f94-e3f126b7af54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:07.906268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bac3c-5581-4e26-9f94-e3f126b7af54 admin admin] PATH_INFO: `/v3/projects/44c01c6c01ff43b99bdd5f17cfc6d723` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:07.907678 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bac3c-5581-4e26-9f94-e3f126b7af54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c01c6c01ff43b99bdd5f17cfc6d723)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:07.911937 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bac3c-5581-4e26-9f94-e3f126b7af54 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:07.966080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-699bac3c-5581-4e26-9f94-e3f126b7af54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c01c6c01ff43b99bdd5f17cfc6d723', 'request_id': 'req-699bac3c-5581-4e26-9f94-e3f126b7af54'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:07.971584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-699bac3c-5581-4e26-9f94-e3f126b7af54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c01c6c01ff43b99bdd5f17cfc6d723', 'request_id': 'req-699bac3c-5581-4e26-9f94-e3f126b7af54'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:07.981394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-699bac3c-5581-4e26-9f94-e3f126b7af54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c01c6c01ff43b99bdd5f17cfc6d723', 'request_id': 'req-699bac3c-5581-4e26-9f94-e3f126b7af54'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:07.994270 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-699bac3c-5581-4e26-9f94-e3f126b7af54 admin admin] The token cache is being invalidate because project 44c01c6c01ff43b99bdd5f17cfc6d723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:38:07.994564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-699bac3c-5581-4e26-9f94-e3f126b7af54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:07.995286 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-699bac3c-5581-4e26-9f94-e3f126b7af54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:38:07.997015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1260/2537] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:38:07 2022] DELETE /identity/v3/projects/44c01c6c01ff43b99bdd5f17cfc6d723 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:08.005859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23340f8d-6f46-48dc-9662-1168baa2fc15 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:08.006010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23340f8d-6f46-48dc-9662-1168baa2fc15 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:08.006278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23340f8d-6f46-48dc-9662-1168baa2fc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:08.030384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-23340f8d-6f46-48dc-9662-1168baa2fc15 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:08.031629 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-23340f8d-6f46-48dc-9662-1168baa2fc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:08.069718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:08.102244 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1278/2538] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:08.116080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d377ca5-a179-4993-9254-08959d1579cc None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:08.116368 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d377ca5-a179-4993-9254-08959d1579cc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:08.116687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d377ca5-a179-4993-9254-08959d1579cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:08.135690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9d377ca5-a179-4993-9254-08959d1579cc None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:08.136933 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9d377ca5-a179-4993-9254-08959d1579cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:08.145446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1261/2539] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:08.149699 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23340f8d-6f46-48dc-9662-1168baa2fc15 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:08.160133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23340f8d-6f46-48dc-9662-1168baa2fc15 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:08.164766 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ceb5b3-e119-4371-bc7b-6b479e26a065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:08.165435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ceb5b3-e119-4371-bc7b-6b479e26a065 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:08.165751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ceb5b3-e119-4371-bc7b-6b479e26a065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:08.166028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ceb5b3-e119-4371-bc7b-6b479e26a065 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:08.166768 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ceb5b3-e119-4371-bc7b-6b479e26a065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:08.167076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ceb5b3-e119-4371-bc7b-6b479e26a065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:08.168683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ceb5b3-e119-4371-bc7b-6b479e26a065 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:08.180768 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1279/2540] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 22:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:08.185199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d377ca5-a179-4993-9254-08959d1579cc None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:08.195217 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d377ca5-a179-4993-9254-08959d1579cc None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:08.199904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f88bf-e704-4c10-8466-d4da1c0e12e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:08.200587 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f88bf-e704-4c10-8466-d4da1c0e12e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:08.200873 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f88bf-e704-4c10-8466-d4da1c0e12e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:08.201156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f88bf-e704-4c10-8466-d4da1c0e12e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:08.201839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f88bf-e704-4c10-8466-d4da1c0e12e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:08.203745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f88bf-e704-4c10-8466-d4da1c0e12e9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:08.230030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1262/2541] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:08 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:08.234074 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ceb5b3-e119-4371-bc7b-6b479e26a065 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:08.244492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ceb5b3-e119-4371-bc7b-6b479e26a065 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:08.301157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0107d3-b518-48e0-b121-38c2d5f647e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:08.301859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0107d3-b518-48e0-b121-38c2d5f647e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:08.302108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0107d3-b518-48e0-b121-38c2d5f647e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:08.302394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0107d3-b518-48e0-b121-38c2d5f647e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:08.303098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0107d3-b518-48e0-b121-38c2d5f647e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:08.305072 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0107d3-b518-48e0-b121-38c2d5f647e2 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:08.326632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1280/2542] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:08 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:08.330884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f88bf-e704-4c10-8466-d4da1c0e12e9 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:08.341889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f88bf-e704-4c10-8466-d4da1c0e12e9 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:08.346543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747fed2-af40-4975-8d72-ada77b00806b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:08.347186 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747fed2-af40-4975-8d72-ada77b00806b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:08.347551 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747fed2-af40-4975-8d72-ada77b00806b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:08.347828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747fed2-af40-4975-8d72-ada77b00806b admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:08.348652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747fed2-af40-4975-8d72-ada77b00806b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:08.350254 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747fed2-af40-4975-8d72-ada77b00806b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:08.360455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1263/2543] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:08.364854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0107d3-b518-48e0-b121-38c2d5f647e2 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:08.374683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0107d3-b518-48e0-b121-38c2d5f647e2 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:08.380157 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d911fe-14c1-42a3-b419-fe4c03e8fe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:08.380932 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d911fe-14c1-42a3-b419-fe4c03e8fe13 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:08.381305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d911fe-14c1-42a3-b419-fe4c03e8fe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:08.381591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d911fe-14c1-42a3-b419-fe4c03e8fe13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:08.382318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d911fe-14c1-42a3-b419-fe4c03e8fe13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:08.383894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d911fe-14c1-42a3-b419-fe4c03e8fe13 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:08.394319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1281/2544] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:08.398526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747fed2-af40-4975-8d72-ada77b00806b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:08.408429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747fed2-af40-4975-8d72-ada77b00806b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:08.413978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f663d7-dd05-4e84-b2f7-513f63702c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:08.414972 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f663d7-dd05-4e84-b2f7-513f63702c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:08.415252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f663d7-dd05-4e84-b2f7-513f63702c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:08.415528 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f663d7-dd05-4e84-b2f7-513f63702c79 admin admin] PATH_INFO: `/v3/projects/2387a046193b4ea98ad71ff3a32eb7e8/users/994c1c9132f146fa84621c6f400f6e8e/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:08.417962 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:08.435264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f663d7-dd05-4e84-b2f7-513f63702c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2387a046193b4ea98ad71ff3a32eb7e8, user_id=994c1c9132f146fa84621c6f400f6e8e, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:08.437982 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f663d7-dd05-4e84-b2f7-513f63702c79 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:08.452905 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1264/2545] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:08 2022] PUT /identity/v3/projects/2387a046193b4ea98ad71ff3a32eb7e8/users/994c1c9132f146fa84621c6f400f6e8e/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:08.458807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f48fed-923c-4caa-a493-8ad856d69638 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:08.459121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f48fed-923c-4caa-a493-8ad856d69638 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:08.459400 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f48fed-923c-4caa-a493-8ad856d69638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:08.477246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-17f48fed-923c-4caa-a493-8ad856d69638 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:08.478441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-17f48fed-923c-4caa-a493-8ad856d69638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:08.537530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1282/2546] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:08.552080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f663d7-dd05-4e84-b2f7-513f63702c79 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:08.554803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-52f663d7-dd05-4e84-b2f7-513f63702c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:08.613769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f663d7-dd05-4e84-b2f7-513f63702c79 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:08.617418 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3dfea-a180-4544-a604-320f6245287b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:08.618064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3dfea-a180-4544-a604-320f6245287b service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:08.618347 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3dfea-a180-4544-a604-320f6245287b service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:08.618622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3dfea-a180-4544-a604-320f6245287b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:08.628023 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3dfea-a180-4544-a604-320f6245287b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:08.630496 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3dfea-a180-4544-a604-320f6245287b service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:08.640000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1265/2547] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:13.457982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb3e90-0be6-4a26-a6aa-6cd42b7ac1b1 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:13.468408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb3e90-0be6-4a26-a6aa-6cd42b7ac1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:13.468772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb3e90-0be6-4a26-a6aa-6cd42b7ac1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:13.508386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-aabb3e90-0be6-4a26-a6aa-6cd42b7ac1b1 None None] MFA Rules not processed for user `994c1c9132f146fa84621c6f400f6e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:13.509750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-aabb3e90-0be6-4a26-a6aa-6cd42b7ac1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:13.557624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:13.597472 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1283/2548] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:38:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:13.598784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3dfea-a180-4544-a604-320f6245287b service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:13.618513 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3dfea-a180-4544-a604-320f6245287b service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:13.625779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d492a-f62d-4d00-b53c-b809ae262409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:13.625779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d492a-f62d-4d00-b53c-b809ae262409 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:13.625779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d492a-f62d-4d00-b53c-b809ae262409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:13.628558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d492a-f62d-4d00-b53c-b809ae262409 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:13.631270 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d492a-f62d-4d00-b53c-b809ae262409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:13.653916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d492a-f62d-4d00-b53c-b809ae262409 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:13.704404 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1266/2549] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 22:38:13 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:13.710880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb3e90-0be6-4a26-a6aa-6cd42b7ac1b1 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:13.724440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb3e90-0be6-4a26-a6aa-6cd42b7ac1b1 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:13.833791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59384d37-cde7-44e3-affb-1588603a6c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:13.834491 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59384d37-cde7-44e3-affb-1588603a6c07 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:13.834850 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59384d37-cde7-44e3-affb-1588603a6c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:13.835134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59384d37-cde7-44e3-affb-1588603a6c07 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:13.835851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59384d37-cde7-44e3-affb-1588603a6c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:13.838210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59384d37-cde7-44e3-affb-1588603a6c07 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:13.864048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1284/2550] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 22:38:13 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:13.868374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d492a-f62d-4d00-b53c-b809ae262409 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:13.882464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d492a-f62d-4d00-b53c-b809ae262409 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:13.889811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87869509-76b7-485a-b233-b7d6730cd6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:13.890968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87869509-76b7-485a-b233-b7d6730cd6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:13.891521 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87869509-76b7-485a-b233-b7d6730cd6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:13.891949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87869509-76b7-485a-b233-b7d6730cd6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:13.892862 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87869509-76b7-485a-b233-b7d6730cd6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:13.895255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87869509-76b7-485a-b233-b7d6730cd6b1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:13.908533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1267/2551] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:13.912844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59384d37-cde7-44e3-affb-1588603a6c07 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:13.923721 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59384d37-cde7-44e3-affb-1588603a6c07 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:13.928886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46a908-278f-4f99-bc8d-3cab560331c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:13.929804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46a908-278f-4f99-bc8d-3cab560331c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:13.930092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46a908-278f-4f99-bc8d-3cab560331c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:13.930376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46a908-278f-4f99-bc8d-3cab560331c2 admin admin] PATH_INFO: `/v3/projects/44cb415b0c1d4816a2797f037309cc7c/users/cf72bbf2e9ff491bbd5cb2dd0988316a/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:13.948663 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46a908-278f-4f99-bc8d-3cab560331c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44cb415b0c1d4816a2797f037309cc7c, user_id=cf72bbf2e9ff491bbd5cb2dd0988316a, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:13.951610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46a908-278f-4f99-bc8d-3cab560331c2 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:13.965944 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1285/2552] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:13 2022] PUT /identity/v3/projects/44cb415b0c1d4816a2797f037309cc7c/users/cf72bbf2e9ff491bbd5cb2dd0988316a/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:13.969846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87869509-76b7-485a-b233-b7d6730cd6b1 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:13.980846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87869509-76b7-485a-b233-b7d6730cd6b1 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:14.044375 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9390a3-8c8c-4461-85ae-691e9242c1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:14.045327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9390a3-8c8c-4461-85ae-691e9242c1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:14.045472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9390a3-8c8c-4461-85ae-691e9242c1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:14.045632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9390a3-8c8c-4461-85ae-691e9242c1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:14.046366 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9390a3-8c8c-4461-85ae-691e9242c1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:14.047974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9390a3-8c8c-4461-85ae-691e9242c1e8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:14.059405 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1268/2553] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 22:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:38:14.063571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46a908-278f-4f99-bc8d-3cab560331c2 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:14.074598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46a908-278f-4f99-bc8d-3cab560331c2 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:14.079459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ee503-dc88-4a6e-9af8-866b4dab385c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:14.080388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ee503-dc88-4a6e-9af8-866b4dab385c admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:14.081217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ee503-dc88-4a6e-9af8-866b4dab385c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:14.081602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ee503-dc88-4a6e-9af8-866b4dab385c admin admin] PATH_INFO: `/v3/projects/44cb415b0c1d4816a2797f037309cc7c/users/cf72bbf2e9ff491bbd5cb2dd0988316a/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:14.083871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ee503-dc88-4a6e-9af8-866b4dab385c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44cb415b0c1d4816a2797f037309cc7c, user_id=cf72bbf2e9ff491bbd5cb2dd0988316a, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:14.086529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ee503-dc88-4a6e-9af8-866b4dab385c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:14.098496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1286/2554] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 22:38:14 2022] PUT /identity/v3/projects/44cb415b0c1d4816a2797f037309cc7c/users/cf72bbf2e9ff491bbd5cb2dd0988316a/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:19.407467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2da08-4e8b-4869-962f-1e4cd3bc47df None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:19.408000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2da08-4e8b-4869-962f-1e4cd3bc47df None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:19.408000 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2da08-4e8b-4869-962f-1e4cd3bc47df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:19.441491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-08f2da08-4e8b-4869-962f-1e4cd3bc47df None None] MFA Rules not processed for user `cf72bbf2e9ff491bbd5cb2dd0988316a`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:19.444602 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-08f2da08-4e8b-4869-962f-1e4cd3bc47df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:19.496855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:19.554410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1269/2555] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:38:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:38:19.568879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ee503-dc88-4a6e-9af8-866b4dab385c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:19.571119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-551ee503-dc88-4a6e-9af8-866b4dab385c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:19.674645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ee503-dc88-4a6e-9af8-866b4dab385c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:19.681235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6726f-d0c0-400a-aa8d-dc13dfbcfcd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:19.682008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6726f-d0c0-400a-aa8d-dc13dfbcfcd7 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:19.682371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6726f-d0c0-400a-aa8d-dc13dfbcfcd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:19.682728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6726f-d0c0-400a-aa8d-dc13dfbcfcd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:19.710162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:19.746396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6726f-d0c0-400a-aa8d-dc13dfbcfcd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:19.748804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6726f-d0c0-400a-aa8d-dc13dfbcfcd7 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:19.761586 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1287/2556] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:24.205909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2da08-4e8b-4869-962f-1e4cd3bc47df None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:24.297645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2da08-4e8b-4869-962f-1e4cd3bc47df None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:24.304204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c649be-e6d3-400d-8533-9255144eec6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:24.304923 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c649be-e6d3-400d-8533-9255144eec6a service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:24.305204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c649be-e6d3-400d-8533-9255144eec6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:24.305493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c649be-e6d3-400d-8533-9255144eec6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:24.315082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e6c649be-e6d3-400d-8533-9255144eec6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:24.404423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c649be-e6d3-400d-8533-9255144eec6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:24.406047 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c649be-e6d3-400d-8533-9255144eec6a service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:24.427041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1270/2557] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:29.755881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=16 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:34.425412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=62 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:49.823052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6726f-d0c0-400a-aa8d-dc13dfbcfcd7 service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:49.834207 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6726f-d0c0-400a-aa8d-dc13dfbcfcd7 service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:49.837800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35112db-a109-4a55-8f63-2b1b93a15800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:49.838458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35112db-a109-4a55-8f63-2b1b93a15800 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:49.838741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35112db-a109-4a55-8f63-2b1b93a15800 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:49.839020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35112db-a109-4a55-8f63-2b1b93a15800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:49.840841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d35112db-a109-4a55-8f63-2b1b93a15800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:49.856652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35112db-a109-4a55-8f63-2b1b93a15800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:49.858132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35112db-a109-4a55-8f63-2b1b93a15800 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:49.870837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1288/2558] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:56.165647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c649be-e6d3-400d-8533-9255144eec6a service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:56.190699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c649be-e6d3-400d-8533-9255144eec6a service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:56.202348 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584944fb-426f-4e03-bdad-39934147d9ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:56.204355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584944fb-426f-4e03-bdad-39934147d9ba service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:56.204355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584944fb-426f-4e03-bdad-39934147d9ba service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:56.204944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584944fb-426f-4e03-bdad-39934147d9ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:56.208226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-584944fb-426f-4e03-bdad-39934147d9ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:56.301189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584944fb-426f-4e03-bdad-39934147d9ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:56.302986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584944fb-426f-4e03-bdad-39934147d9ba service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:56.313742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1271/2559] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:38:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:38:59.865717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=6 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:04.256755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35112db-a109-4a55-8f63-2b1b93a15800 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:04.258253 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d35112db-a109-4a55-8f63-2b1b93a15800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:04.273041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35112db-a109-4a55-8f63-2b1b93a15800 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:04.278500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52bc1d-5064-452e-9cba-c4b22a2a2a8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:04.279139 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52bc1d-5064-452e-9cba-c4b22a2a2a8b service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:04.279422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52bc1d-5064-452e-9cba-c4b22a2a2a8b service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:04.279697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52bc1d-5064-452e-9cba-c4b22a2a2a8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:04.282497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fa52bc1d-5064-452e-9cba-c4b22a2a2a8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:04.344553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52bc1d-5064-452e-9cba-c4b22a2a2a8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:04.346142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52bc1d-5064-452e-9cba-c4b22a2a2a8b service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:04.355269 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1289/2560] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:06.308471 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=28 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:14.350933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=28 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:15.214287 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b9465-9bfc-4fa9-9a19-b896f125b857 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:15.214918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b9465-9bfc-4fa9-9a19-b896f125b857 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:15.214918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b9465-9bfc-4fa9-9a19-b896f125b857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:15.236014 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f51b9465-9bfc-4fa9-9a19-b896f125b857 None None] MFA Rules not processed for user `f93bd44f2fbc4be4be905c8b8ae0e343`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:15.237688 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f51b9465-9bfc-4fa9-9a19-b896f125b857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:15.245297 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1272/2561] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:39:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:39:15.355088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52bc1d-5064-452e-9cba-c4b22a2a2a8b service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:15.357493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-fa52bc1d-5064-452e-9cba-c4b22a2a2a8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:15.451818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52bc1d-5064-452e-9cba-c4b22a2a2a8b service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:15.456118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e8bf1-5f2b-442e-9ca8-88e207595ba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:15.456818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e8bf1-5f2b-442e-9ca8-88e207595ba7 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:15.457106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e8bf1-5f2b-442e-9ca8-88e207595ba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:15.457387 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e8bf1-5f2b-442e-9ca8-88e207595ba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:15.459348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b54e8bf1-5f2b-442e-9ca8-88e207595ba7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:15.475807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e8bf1-5f2b-442e-9ca8-88e207595ba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:15.477312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e8bf1-5f2b-442e-9ca8-88e207595ba7 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:15.490177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1290/2562] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:22.347857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b9465-9bfc-4fa9-9a19-b896f125b857 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:22.360226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b9465-9bfc-4fa9-9a19-b896f125b857 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:22.365143 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7639c7-0a87-4e2a-8b5b-ae07d7198b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:22.365957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7639c7-0a87-4e2a-8b5b-ae07d7198b15 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:22.366429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7639c7-0a87-4e2a-8b5b-ae07d7198b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:22.366863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7639c7-0a87-4e2a-8b5b-ae07d7198b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:22.369428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9a7639c7-0a87-4e2a-8b5b-ae07d7198b15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:22.382557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7639c7-0a87-4e2a-8b5b-ae07d7198b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:22.384052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7639c7-0a87-4e2a-8b5b-ae07d7198b15 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:22.394699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1273/2563] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:24.298531 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e8bf1-5f2b-442e-9ca8-88e207595ba7 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:24.310130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e8bf1-5f2b-442e-9ca8-88e207595ba7 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:24.316001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557b356-b42b-4ff6-ba4e-51a3a0c0c1e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:24.316686 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557b356-b42b-4ff6-ba4e-51a3a0c0c1e7 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:24.316972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557b356-b42b-4ff6-ba4e-51a3a0c0c1e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:24.317251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557b356-b42b-4ff6-ba4e-51a3a0c0c1e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:24.319042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-f557b356-b42b-4ff6-ba4e-51a3a0c0c1e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:24.333833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557b356-b42b-4ff6-ba4e-51a3a0c0c1e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:24.335641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557b356-b42b-4ff6-ba4e-51a3a0c0c1e7 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:24.349141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1291/2564] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:39:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:39:32.389280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=9 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:34.343869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=32 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:40:42.477322 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7639c7-0a87-4e2a-8b5b-ae07d7198b15 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:40:42.491533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7639c7-0a87-4e2a-8b5b-ae07d7198b15 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:40:42.497850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844e16b-e103-41b0-99fe-fd4d17cc0f91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:40:42.498790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844e16b-e103-41b0-99fe-fd4d17cc0f91 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:40:42.499078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844e16b-e103-41b0-99fe-fd4d17cc0f91 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:40:42.499361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844e16b-e103-41b0-99fe-fd4d17cc0f91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:40:42.501393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9844e16b-e103-41b0-99fe-fd4d17cc0f91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:40:42.515977 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844e16b-e103-41b0-99fe-fd4d17cc0f91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:40:42.517842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844e16b-e103-41b0-99fe-fd4d17cc0f91 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:40:42.530504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1274/2565] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:40:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:40:52.525014 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=6 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:42:20.051321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557b356-b42b-4ff6-ba4e-51a3a0c0c1e7 service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:42:20.119761 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557b356-b42b-4ff6-ba4e-51a3a0c0c1e7 service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:42:20.125869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1f8f4-fe82-4eab-b127-c1fab94336a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:42:20.126548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1f8f4-fe82-4eab-b127-c1fab94336a9 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:42:20.126833 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1f8f4-fe82-4eab-b127-c1fab94336a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:42:20.127108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1f8f4-fe82-4eab-b127-c1fab94336a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:42:20.128927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6ec1f8f4-fe82-4eab-b127-c1fab94336a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:42:20.195416 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1f8f4-fe82-4eab-b127-c1fab94336a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:42:20.197111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1f8f4-fe82-4eab-b127-c1fab94336a9 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:42:20.206324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1292/2566] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:42:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:42:30.202116 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=56 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:42:39.872129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844e16b-e103-41b0-99fe-fd4d17cc0f91 service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:42:39.913140 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844e16b-e103-41b0-99fe-fd4d17cc0f91 service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:42:39.916912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435e487-7f35-44f8-bbbd-b86e2356f903 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:42:39.917564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435e487-7f35-44f8-bbbd-b86e2356f903 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:42:39.917846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435e487-7f35-44f8-bbbd-b86e2356f903 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:42:39.918122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435e487-7f35-44f8-bbbd-b86e2356f903 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:42:39.919918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8435e487-7f35-44f8-bbbd-b86e2356f903 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:42:39.987271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435e487-7f35-44f8-bbbd-b86e2356f903 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:42:39.988742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435e487-7f35-44f8-bbbd-b86e2356f903 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:42:39.997962 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1275/2567] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:42:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:42:49.644029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1f8f4-fe82-4eab-b127-c1fab94336a9 service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:42:49.645615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6ec1f8f4-fe82-4eab-b127-c1fab94336a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:42:49.741361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1f8f4-fe82-4eab-b127-c1fab94336a9 service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:42:49.746295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115a1a9-c3fc-490a-858d-d0797baab51b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:42:49.746945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115a1a9-c3fc-490a-858d-d0797baab51b service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:42:49.747228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115a1a9-c3fc-490a-858d-d0797baab51b service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:42:49.747505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115a1a9-c3fc-490a-858d-d0797baab51b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:42:49.749554 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9115a1a9-c3fc-490a-858d-d0797baab51b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:42:49.761641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115a1a9-c3fc-490a-858d-d0797baab51b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:42:49.763196 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115a1a9-c3fc-490a-858d-d0797baab51b service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:42:49.773747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1293/2568] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:42:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:42:49.993447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=40 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:42:59.769693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=41 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:43:50.941120 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435e487-7f35-44f8-bbbd-b86e2356f903 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:50.960996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435e487-7f35-44f8-bbbd-b86e2356f903 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:50.965044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3855ab-955b-4876-a49c-b0c7cdb695ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:50.965686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3855ab-955b-4876-a49c-b0c7cdb695ef service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:50.965970 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3855ab-955b-4876-a49c-b0c7cdb695ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:50.966248 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3855ab-955b-4876-a49c-b0c7cdb695ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:50.976863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3855ab-955b-4876-a49c-b0c7cdb695ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:50.978876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3855ab-955b-4876-a49c-b0c7cdb695ef service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:50.990374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1276/2569] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:43:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:43:56.084091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115a1a9-c3fc-490a-858d-d0797baab51b service placement] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:43:56.100639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115a1a9-c3fc-490a-858d-d0797baab51b service placement] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:43:56.106990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62091b8f-6d6f-4b33-92d1-91a8939c15e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:43:56.107649 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62091b8f-6d6f-4b33-92d1-91a8939c15e0 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:43:56.107926 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62091b8f-6d6f-4b33-92d1-91a8939c15e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:43:56.108201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62091b8f-6d6f-4b33-92d1-91a8939c15e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:43:56.117277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62091b8f-6d6f-4b33-92d1-91a8939c15e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:43:56.118872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62091b8f-6d6f-4b33-92d1-91a8939c15e0 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:43:56.148914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1294/2570] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:43:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:44:00.985233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=10 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:04.443290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3855ab-955b-4876-a49c-b0c7cdb695ef service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:44:04.455437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3855ab-955b-4876-a49c-b0c7cdb695ef service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:44:04.461241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709e2fa-09dd-4546-8890-61b83a5603be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:44:04.462450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709e2fa-09dd-4546-8890-61b83a5603be service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:04.462756 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709e2fa-09dd-4546-8890-61b83a5603be service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:04.463387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709e2fa-09dd-4546-8890-61b83a5603be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:04.471714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709e2fa-09dd-4546-8890-61b83a5603be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:44:04.473429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709e2fa-09dd-4546-8890-61b83a5603be service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:44:04.502542 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1277/2571] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:44:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:44:06.145974 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=13 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:14.500065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=13 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:15.609280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62091b8f-6d6f-4b33-92d1-91a8939c15e0 service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:44:15.610691 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-62091b8f-6d6f-4b33-92d1-91a8939c15e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:15.634602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62091b8f-6d6f-4b33-92d1-91a8939c15e0 service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:44:15.638200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b44aa3-e417-40e4-ac36-79d626832b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:44:15.638837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b44aa3-e417-40e4-ac36-79d626832b13 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:15.639126 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b44aa3-e417-40e4-ac36-79d626832b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:15.639399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b44aa3-e417-40e4-ac36-79d626832b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:15.650499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b44aa3-e417-40e4-ac36-79d626832b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:44:15.651940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b44aa3-e417-40e4-ac36-79d626832b13 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:44:15.663821 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1295/2572] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:44:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:44:25.658824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=11 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:40.115768 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709e2fa-09dd-4546-8890-61b83a5603be service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:44:40.125156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709e2fa-09dd-4546-8890-61b83a5603be service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:44:40.129391 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b208c-0d2b-4049-8754-680a71be402f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:44:40.130033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b208c-0d2b-4049-8754-680a71be402f service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:40.130315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b208c-0d2b-4049-8754-680a71be402f service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:40.130590 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b208c-0d2b-4049-8754-680a71be402f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:40.138900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b208c-0d2b-4049-8754-680a71be402f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:44:40.140320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b208c-0d2b-4049-8754-680a71be402f service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:44:40.149558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1278/2573] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:44:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:44:45.298602 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b44aa3-e417-40e4-ac36-79d626832b13 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:44:45.307347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b44aa3-e417-40e4-ac36-79d626832b13 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:44:45.311063 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf5812-62d9-4f0f-96ca-fd6e5cff1d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:44:45.311705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf5812-62d9-4f0f-96ca-fd6e5cff1d3b service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:45.311992 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf5812-62d9-4f0f-96ca-fd6e5cff1d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:45.312276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf5812-62d9-4f0f-96ca-fd6e5cff1d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:45.320369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf5812-62d9-4f0f-96ca-fd6e5cff1d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:44:45.321842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf5812-62d9-4f0f-96ca-fd6e5cff1d3b service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:44:45.330137 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1296/2574] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:44:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:44:50.145596 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=6 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:55.326371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=6 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:13.805626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b208c-0d2b-4049-8754-680a71be402f service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:13.819369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b208c-0d2b-4049-8754-680a71be402f service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:13.824898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826399c2-c421-4187-8b4f-e9427ead87d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:13.825632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826399c2-c421-4187-8b4f-e9427ead87d7 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:13.825914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826399c2-c421-4187-8b4f-e9427ead87d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:13.826189 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826399c2-c421-4187-8b4f-e9427ead87d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:13.828627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-826399c2-c421-4187-8b4f-e9427ead87d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:13.854790 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826399c2-c421-4187-8b4f-e9427ead87d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:13.856369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826399c2-c421-4187-8b4f-e9427ead87d7 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:13.866979 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1279/2575] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:21.043141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf5812-62d9-4f0f-96ca-fd6e5cff1d3b service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:21.057231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf5812-62d9-4f0f-96ca-fd6e5cff1d3b service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:21.063446 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf35c9f-1ead-4731-b0ba-87d8fce444f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:21.064132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35c9f-1ead-4731-b0ba-87d8fce444f4 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:21.064408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35c9f-1ead-4731-b0ba-87d8fce444f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:21.064713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35c9f-1ead-4731-b0ba-87d8fce444f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:21.072500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf35c9f-1ead-4731-b0ba-87d8fce444f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:21.074096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf35c9f-1ead-4731-b0ba-87d8fce444f4 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:21.102475 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1297/2576] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:23.862502 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=11 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:31.099807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=13 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:40.456986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826399c2-c421-4187-8b4f-e9427ead87d7 service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:40.474624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826399c2-c421-4187-8b4f-e9427ead87d7 service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:40.480493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fd41a-e994-4dd5-b772-2895acb2bdbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:40.481154 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fd41a-e994-4dd5-b772-2895acb2bdbb service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:40.481439 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fd41a-e994-4dd5-b772-2895acb2bdbb service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:40.481717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fd41a-e994-4dd5-b772-2895acb2bdbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:40.490122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fd41a-e994-4dd5-b772-2895acb2bdbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:40.491708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fd41a-e994-4dd5-b772-2895acb2bdbb service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:40.519700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1280/2577] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:50.516450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=13 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:53.678210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf35c9f-1ead-4731-b0ba-87d8fce444f4 service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:53.679991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2bf35c9f-1ead-4731-b0ba-87d8fce444f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:53.697169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf35c9f-1ead-4731-b0ba-87d8fce444f4 service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:53.704962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb2f46-01ec-422f-b161-1eae4aacba2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:53.705644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb2f46-01ec-422f-b161-1eae4aacba2f service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:53.705928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb2f46-01ec-422f-b161-1eae4aacba2f service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:53.706210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb2f46-01ec-422f-b161-1eae4aacba2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:53.727043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb2f46-01ec-422f-b161-1eae4aacba2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:53.728741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb2f46-01ec-422f-b161-1eae4aacba2f service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:53.740653 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1298/2578] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:48:03.735399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=6 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:13.881161 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fd41a-e994-4dd5-b772-2895acb2bdbb service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:13.881161 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-697fd41a-e994-4dd5-b772-2895acb2bdbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:13.912868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fd41a-e994-4dd5-b772-2895acb2bdbb service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:13.917098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9448522-1423-4a1e-a98b-746d4d321089 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:13.917884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9448522-1423-4a1e-a98b-746d4d321089 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:13.918112 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9448522-1423-4a1e-a98b-746d4d321089 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:13.918388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9448522-1423-4a1e-a98b-746d4d321089 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:13.920506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c9448522-1423-4a1e-a98b-746d4d321089 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:13.998463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9448522-1423-4a1e-a98b-746d4d321089 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:13.999930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9448522-1423-4a1e-a98b-746d4d321089 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:14.013547 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1281/2579] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:49:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:49:14.065029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb2f46-01ec-422f-b161-1eae4aacba2f service placement] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:14.075070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb2f46-01ec-422f-b161-1eae4aacba2f service placement] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:14.078670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba22274-9c8b-413e-80e7-0a77cf3e999a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:14.079320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba22274-9c8b-413e-80e7-0a77cf3e999a service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:14.079605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba22274-9c8b-413e-80e7-0a77cf3e999a service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:14.079915 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba22274-9c8b-413e-80e7-0a77cf3e999a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:14.081844 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-eba22274-9c8b-413e-80e7-0a77cf3e999a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:14.154615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba22274-9c8b-413e-80e7-0a77cf3e999a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:14.155984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba22274-9c8b-413e-80e7-0a77cf3e999a service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:14.165304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1299/2580] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:49:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:49:14.249539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9448522-1423-4a1e-a98b-746d4d321089 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:14.258436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9448522-1423-4a1e-a98b-746d4d321089 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:14.262089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95337133-092d-4b94-903e-738733aa8a9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:14.262734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95337133-092d-4b94-903e-738733aa8a9e service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:14.263020 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95337133-092d-4b94-903e-738733aa8a9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:14.263299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95337133-092d-4b94-903e-738733aa8a9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:14.265200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-95337133-092d-4b94-903e-738733aa8a9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:14.339461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95337133-092d-4b94-903e-738733aa8a9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:14.340956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95337133-092d-4b94-903e-738733aa8a9e service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:14.350091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1282/2581] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:49:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:49:17.459009 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba22274-9c8b-413e-80e7-0a77cf3e999a service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:17.513686 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba22274-9c8b-413e-80e7-0a77cf3e999a service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:17.517320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b8a6f-5a92-4911-b926-0aeb4584e1c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:17.517963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b8a6f-5a92-4911-b926-0aeb4584e1c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:17.518246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b8a6f-5a92-4911-b926-0aeb4584e1c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:17.518523 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b8a6f-5a92-4911-b926-0aeb4584e1c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:17.520397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bd0b8a6f-5a92-4911-b926-0aeb4584e1c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:17.533899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b8a6f-5a92-4911-b926-0aeb4584e1c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:17.535313 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b8a6f-5a92-4911-b926-0aeb4584e1c8 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:17.546725 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1300/2582] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:49:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:49:24.346571 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=72 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:27.541921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=59 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:50:41.073699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95337133-092d-4b94-903e-738733aa8a9e service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:50:41.075295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-95337133-092d-4b94-903e-738733aa8a9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:50:41.132200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95337133-092d-4b94-903e-738733aa8a9e service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:50:41.136372 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8e066-310e-4924-99a3-d7fe7270b6e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:50:41.137045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8e066-310e-4924-99a3-d7fe7270b6e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:50:41.137327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8e066-310e-4924-99a3-d7fe7270b6e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:50:41.137600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8e066-310e-4924-99a3-d7fe7270b6e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:50:41.139493 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-80c8e066-310e-4924-99a3-d7fe7270b6e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:50:41.220772 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8e066-310e-4924-99a3-d7fe7270b6e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:50:41.222214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8e066-310e-4924-99a3-d7fe7270b6e9 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:50:41.240990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1283/2583] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:50:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:50:51.238198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=61 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:50:52.308201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b8a6f-5a92-4911-b926-0aeb4584e1c8 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:50:52.309952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bd0b8a6f-5a92-4911-b926-0aeb4584e1c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:50:52.320695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b8a6f-5a92-4911-b926-0aeb4584e1c8 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:50:52.324328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a200817-b856-443b-a27d-f1103f0dbd1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:50:52.325004 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a200817-b856-443b-a27d-f1103f0dbd1b service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:50:52.325284 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a200817-b856-443b-a27d-f1103f0dbd1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:50:52.325561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a200817-b856-443b-a27d-f1103f0dbd1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:50:52.327361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3a200817-b856-443b-a27d-f1103f0dbd1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:50:52.336634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a200817-b856-443b-a27d-f1103f0dbd1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:50:52.338689 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a200817-b856-443b-a27d-f1103f0dbd1b service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:50:52.346621 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1301/2584] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:50:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:51:02.343012 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=6 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:51:14.978105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8e066-310e-4924-99a3-d7fe7270b6e9 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:14.989059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8e066-310e-4924-99a3-d7fe7270b6e9 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:14.993681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8191288-4f19-4560-a71e-93f0c7534e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:14.994310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8191288-4f19-4560-a71e-93f0c7534e4c service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:14.994593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8191288-4f19-4560-a71e-93f0c7534e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:14.994870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8191288-4f19-4560-a71e-93f0c7534e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:14.996842 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c8191288-4f19-4560-a71e-93f0c7534e4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:15.085485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8191288-4f19-4560-a71e-93f0c7534e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:15.086968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8191288-4f19-4560-a71e-93f0c7534e4c service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:15.106031 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1284/2585] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:51:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:51:18.282967 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a200817-b856-443b-a27d-f1103f0dbd1b service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:18.299551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a200817-b856-443b-a27d-f1103f0dbd1b service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:18.308128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88577450-9f39-4303-b4f8-e6a761760592 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:18.309141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88577450-9f39-4303-b4f8-e6a761760592 service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:18.309778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88577450-9f39-4303-b4f8-e6a761760592 service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:18.309972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88577450-9f39-4303-b4f8-e6a761760592 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:18.311905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-88577450-9f39-4303-b4f8-e6a761760592 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:18.328628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88577450-9f39-4303-b4f8-e6a761760592 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:18.329763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88577450-9f39-4303-b4f8-e6a761760592 service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:18.352784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1302/2586] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:51:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:51:21.687209 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8191288-4f19-4560-a71e-93f0c7534e4c service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:21.688858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c8191288-4f19-4560-a71e-93f0c7534e4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:21.702768 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8191288-4f19-4560-a71e-93f0c7534e4c service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:21.707700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:21.708245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:21.708559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:21.708860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] PATH_INFO: `/v3/users/994c1c9132f146fa84621c6f400f6e8e` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:21.710071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=994c1c9132f146fa84621c6f400f6e8e)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:21.713037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:21.762616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994c1c9132f146fa84621c6f400f6e8e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:21.769584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994c1c9132f146fa84621c6f400f6e8e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:21.781401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994c1c9132f146fa84621c6f400f6e8e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:21.789063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994c1c9132f146fa84621c6f400f6e8e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:21.790388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994c1c9132f146fa84621c6f400f6e8e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:21.796714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1285/2587] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:51:21 2022] DELETE /identity/v3/users/994c1c9132f146fa84621c6f400f6e8e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:51:21.972377 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88577450-9f39-4303-b4f8-e6a761760592 service placement] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:21.983627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88577450-9f39-4303-b4f8-e6a761760592 service placement] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:22.044527 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:22.045740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:22.046025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:22.046301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] PATH_INFO: `/v3/projects/2387a046193b4ea98ad71ff3a32eb7e8` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:22.047482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2387a046193b4ea98ad71ff3a32eb7e8)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:22.049413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:22.096240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2387a046193b4ea98ad71ff3a32eb7e8', 'request_id': 'req-ee555b0e-987e-4de8-b199-4813d3ecc510'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.103753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2387a046193b4ea98ad71ff3a32eb7e8', 'request_id': 'req-ee555b0e-987e-4de8-b199-4813d3ecc510'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.109574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2387a046193b4ea98ad71ff3a32eb7e8', 'request_id': 'req-ee555b0e-987e-4de8-b199-4813d3ecc510'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.123492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] The token cache is being invalidate because project 2387a046193b4ea98ad71ff3a32eb7e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:51:22.123914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.124615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.126248 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1303/2588] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:51:21 2022] DELETE /identity/v3/projects/2387a046193b4ea98ad71ff3a32eb7e8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:51:22.130302 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:22.131729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:22.145953 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06211992-8efa-4261-92dd-fbdb66193d71 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:22.182891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:51:22.212268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:22.213190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:22.213484 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:22.213764 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] PATH_INFO: `/v3/users/cf72bbf2e9ff491bbd5cb2dd0988316a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:22.228958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf72bbf2e9ff491bbd5cb2dd0988316a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:22.230922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:22.280340 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf72bbf2e9ff491bbd5cb2dd0988316a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.287152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf72bbf2e9ff491bbd5cb2dd0988316a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.299931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf72bbf2e9ff491bbd5cb2dd0988316a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.307896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf72bbf2e9ff491bbd5cb2dd0988316a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.308697 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf72bbf2e9ff491bbd5cb2dd0988316a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.315033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1286/2589] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:51:22 2022] DELETE /identity/v3/users/cf72bbf2e9ff491bbd5cb2dd0988316a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:51:22.493789 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:22.495336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:22.514085 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee555b0e-987e-4de8-b199-4813d3ecc510 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:22.575821 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:22.576673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:22.576964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:22.577320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] PATH_INFO: `/v3/projects/44cb415b0c1d4816a2797f037309cc7c` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:22.588371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44cb415b0c1d4816a2797f037309cc7c)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:22.590742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:22.603290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:51:22.640747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44cb415b0c1d4816a2797f037309cc7c', 'request_id': 'req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.648764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44cb415b0c1d4816a2797f037309cc7c', 'request_id': 'req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.655032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44cb415b0c1d4816a2797f037309cc7c', 'request_id': 'req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.668537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] The token cache is being invalidate because project 44cb415b0c1d4816a2797f037309cc7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:51:22.668828 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.669488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:51:22.671070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1304/2590] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:51:22 2022] DELETE /identity/v3/projects/44cb415b0c1d4816a2797f037309cc7c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:51:32.311393 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=41,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:51:32.665137 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:52:20.037321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:52:20.038768 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:52:20.147861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f828a-31da-41f1-a33b-4420106e8a27 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:52:20.216226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb3973-d06a-42ee-980c-6e10c39ef2bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:52:20.216914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb3973-d06a-42ee-980c-6e10c39ef2bd service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:52:20.217343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb3973-d06a-42ee-980c-6e10c39ef2bd service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:52:20.217618 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb3973-d06a-42ee-980c-6e10c39ef2bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:52:20.219546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-60fb3973-d06a-42ee-980c-6e10c39ef2bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:52:20.285789 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:52:20.303708 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb3973-d06a-42ee-980c-6e10c39ef2bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:52:20.305353 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb3973-d06a-42ee-980c-6e10c39ef2bd service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:52:20.314311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1287/2591] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:52:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:52:30.310147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=9 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:00.601461 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:00.602865 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:00.733029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d54d5-5a87-4d3b-9ea3-a2cd9f9a0639 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:00.738301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c20a2-4e2c-4f6c-b98b-e4bd2212b8ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:00.738949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c20a2-4e2c-4f6c-b98b-e4bd2212b8ad service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:00.739234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c20a2-4e2c-4f6c-b98b-e4bd2212b8ad service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:00.739512 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c20a2-4e2c-4f6c-b98b-e4bd2212b8ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:00.741329 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b31c20a2-4e2c-4f6c-b98b-e4bd2212b8ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:00.753084 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c20a2-4e2c-4f6c-b98b-e4bd2212b8ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:00.754635 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c20a2-4e2c-4f6c-b98b-e4bd2212b8ad service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:00.765403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1305/2592] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:10.760537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=59 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:14.633463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb3973-d06a-42ee-980c-6e10c39ef2bd service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:14.634933 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-60fb3973-d06a-42ee-980c-6e10c39ef2bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:14.726445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb3973-d06a-42ee-980c-6e10c39ef2bd service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:14.730455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a34314-da8a-461c-8ef1-f65fbb80d006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:14.731102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a34314-da8a-461c-8ef1-f65fbb80d006 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:14.731389 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a34314-da8a-461c-8ef1-f65fbb80d006 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:14.731669 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a34314-da8a-461c-8ef1-f65fbb80d006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:14.734119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-89a34314-da8a-461c-8ef1-f65fbb80d006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:14.769530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c20a2-4e2c-4f6c-b98b-e4bd2212b8ad service placement] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:14.771168 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b31c20a2-4e2c-4f6c-b98b-e4bd2212b8ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:14.799843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a34314-da8a-461c-8ef1-f65fbb80d006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:14.801495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a34314-da8a-461c-8ef1-f65fbb80d006 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:14.811709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1288/2593] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:14.844302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c20a2-4e2c-4f6c-b98b-e4bd2212b8ad service placement] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:14.847967 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b566fc-184e-489c-85c8-2734a462fcd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:14.848652 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b566fc-184e-489c-85c8-2734a462fcd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:14.848958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b566fc-184e-489c-85c8-2734a462fcd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:14.849237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b566fc-184e-489c-85c8-2734a462fcd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:14.851088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-77b566fc-184e-489c-85c8-2734a462fcd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:14.928524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b566fc-184e-489c-85c8-2734a462fcd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:14.929940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b566fc-184e-489c-85c8-2734a462fcd8 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:14.940290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1306/2594] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:53:18.045135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a34314-da8a-461c-8ef1-f65fbb80d006 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:18.047247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-89a34314-da8a-461c-8ef1-f65fbb80d006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:18.061869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a34314-da8a-461c-8ef1-f65fbb80d006 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:18.066917 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73371b08-2e0d-470a-be98-bf7701b980f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:18.067643 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73371b08-2e0d-470a-be98-bf7701b980f4 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:18.068066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73371b08-2e0d-470a-be98-bf7701b980f4 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:18.068395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73371b08-2e0d-470a-be98-bf7701b980f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:18.070513 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-73371b08-2e0d-470a-be98-bf7701b980f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:18.085319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73371b08-2e0d-470a-be98-bf7701b980f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:18.087427 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73371b08-2e0d-470a-be98-bf7701b980f4 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:18.101893 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1289/2595] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:21.236538 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b566fc-184e-489c-85c8-2734a462fcd8 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:21.238884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-77b566fc-184e-489c-85c8-2734a462fcd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:21.259986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b566fc-184e-489c-85c8-2734a462fcd8 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:21.265151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:21.266071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:21.266357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:21.266640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] PATH_INFO: `/v3/users/68f8c3b4d761425db457093edfccce6b` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:21.267869 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68f8c3b4d761425db457093edfccce6b)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:21.269909 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:21.323399 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68f8c3b4d761425db457093edfccce6b'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.331647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68f8c3b4d761425db457093edfccce6b'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.332332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68f8c3b4d761425db457093edfccce6b'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.339388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68f8c3b4d761425db457093edfccce6b'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.340210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68f8c3b4d761425db457093edfccce6b'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.358460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1307/2596] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:21 2022] DELETE /identity/v3/users/68f8c3b4d761425db457093edfccce6b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:21.572164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73371b08-2e0d-470a-be98-bf7701b980f4 service placement] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:21.582963 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73371b08-2e0d-470a-be98-bf7701b980f4 service placement] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:21.641202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:21.642069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:21.642353 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:21.642632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] PATH_INFO: `/v3/projects/671dbc43a5c04a26b23e278f3f7964bb` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:21.643814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=671dbc43a5c04a26b23e278f3f7964bb)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:21.645772 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:21.667510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:21.693061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '671dbc43a5c04a26b23e278f3f7964bb', 'request_id': 'req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.698562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '671dbc43a5c04a26b23e278f3f7964bb', 'request_id': 'req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.706419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '671dbc43a5c04a26b23e278f3f7964bb', 'request_id': 'req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.718155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] The token cache is being invalidate because project 671dbc43a5c04a26b23e278f3f7964bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:21.718438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.719064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.720616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1290/2597] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:21 2022] DELETE /identity/v3/projects/671dbc43a5c04a26b23e278f3f7964bb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:21.724531 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:21.725931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:21.739478 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d4011-2be6-4ebb-ba2d-9b8913599cd1 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:21.743080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:21.797731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:21.798607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:21.798887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:21.799161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] PATH_INFO: `/v3/users/7f396e9269b046db8d9eec2e86ecd534` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:21.812509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f396e9269b046db8d9eec2e86ecd534)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:21.814440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:21.860444 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f396e9269b046db8d9eec2e86ecd534'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.868898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f396e9269b046db8d9eec2e86ecd534'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.869525 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f396e9269b046db8d9eec2e86ecd534'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.876816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f396e9269b046db8d9eec2e86ecd534'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.877426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f396e9269b046db8d9eec2e86ecd534'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.894959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1308/2598] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:21 2022] DELETE /identity/v3/users/7f396e9269b046db8d9eec2e86ecd534 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:22.078499 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:22.080032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:22.095498 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f9bfe-1c5b-46a5-acf1-50c9abfb01e5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:22.153397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:22.154264 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:22.154547 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:22.155476 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] PATH_INFO: `/v3/projects/617c3f22038947e3a09375406705efdb` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:22.166160 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=617c3f22038947e3a09375406705efdb)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:22.168099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:22.214432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '617c3f22038947e3a09375406705efdb', 'request_id': 'req-2b0dcc4b-6cf4-4130-bfdd-847ead466169'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:22.220718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '617c3f22038947e3a09375406705efdb', 'request_id': 'req-2b0dcc4b-6cf4-4130-bfdd-847ead466169'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:22.228217 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '617c3f22038947e3a09375406705efdb', 'request_id': 'req-2b0dcc4b-6cf4-4130-bfdd-847ead466169'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:22.241945 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] The token cache is being invalidate because project 617c3f22038947e3a09375406705efdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:22.242233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:22.242867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:22.244424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1291/2599] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:22 2022] DELETE /identity/v3/projects/617c3f22038947e3a09375406705efdb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:31.891415 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:32.239040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=71,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:14.387792 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:14.389323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:14.476731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84087a31-3e3f-4061-8955-002a648627e7 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:14.540402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f2ce6-24f6-41c8-b37c-d18c95a4833e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:14.541108 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f2ce6-24f6-41c8-b37c-d18c95a4833e service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:14.541397 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f2ce6-24f6-41c8-b37c-d18c95a4833e service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:14.541678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f2ce6-24f6-41c8-b37c-d18c95a4833e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:14.543522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9e7f2ce6-24f6-41c8-b37c-d18c95a4833e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:14.615285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f2ce6-24f6-41c8-b37c-d18c95a4833e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:14.616961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f2ce6-24f6-41c8-b37c-d18c95a4833e service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:14.636441 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1309/2600] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:15.787607 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:15.789141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:15.864775 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dcc4b-6cf4-4130-bfdd-847ead466169 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:15.868345 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bbdfe-5433-452b-ab81-e1680bf4f74a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:15.869079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bbdfe-5433-452b-ab81-e1680bf4f74a service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:15.869362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bbdfe-5433-452b-ab81-e1680bf4f74a service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:15.869635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bbdfe-5433-452b-ab81-e1680bf4f74a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:15.871422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-734bbdfe-5433-452b-ab81-e1680bf4f74a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:15.885116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bbdfe-5433-452b-ab81-e1680bf4f74a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:15.886536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bbdfe-5433-452b-ab81-e1680bf4f74a service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:15.897909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1292/2601] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:17.007142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f2ce6-24f6-41c8-b37c-d18c95a4833e service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:17.008986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9e7f2ce6-24f6-41c8-b37c-d18c95a4833e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:17.020920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f2ce6-24f6-41c8-b37c-d18c95a4833e service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:17.024486 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9c560-114f-41ad-99e2-fcd731ab625f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:17.025142 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9c560-114f-41ad-99e2-fcd731ab625f service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:17.025454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9c560-114f-41ad-99e2-fcd731ab625f service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:17.025733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9c560-114f-41ad-99e2-fcd731ab625f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:17.027450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-83c9c560-114f-41ad-99e2-fcd731ab625f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:17.041677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9c560-114f-41ad-99e2-fcd731ab625f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:17.043087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9c560-114f-41ad-99e2-fcd731ab625f service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:17.054590 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1310/2602] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:25.893606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=35 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:27.049675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=98 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:15.188111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bbdfe-5433-452b-ab81-e1680bf4f74a service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:15.189549 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-734bbdfe-5433-452b-ab81-e1680bf4f74a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:15.245222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bbdfe-5433-452b-ab81-e1680bf4f74a service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:15.248791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549652d4-5eed-4ce9-8886-abf22836526e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:15.249435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549652d4-5eed-4ce9-8886-abf22836526e service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:15.249712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549652d4-5eed-4ce9-8886-abf22836526e service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:15.249984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549652d4-5eed-4ce9-8886-abf22836526e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:15.251752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-549652d4-5eed-4ce9-8886-abf22836526e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:15.322335 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549652d4-5eed-4ce9-8886-abf22836526e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:15.323783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549652d4-5eed-4ce9-8886-abf22836526e service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:15.342504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1293/2603] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:55:25.339848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=56 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:59.266823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9c560-114f-41ad-99e2-fcd731ab625f service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:59.275886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9c560-114f-41ad-99e2-fcd731ab625f service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:59.279538 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea0699-8239-4608-861e-a36bb3407cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:59.280183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea0699-8239-4608-861e-a36bb3407cac service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:59.280469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea0699-8239-4608-861e-a36bb3407cac service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:59.280785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea0699-8239-4608-861e-a36bb3407cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:59.283024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-62ea0699-8239-4608-861e-a36bb3407cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:59.294020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea0699-8239-4608-861e-a36bb3407cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:59.295453 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea0699-8239-4608-861e-a36bb3407cac service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:59.305014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1311/2604] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:56:09.300066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=6 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:43.658192 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fe024-5085-4f8a-bfdc-3b49a65bf8a7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:43.658737 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fe024-5085-4f8a-bfdc-3b49a65bf8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:43.658852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6fe024-5085-4f8a-bfdc-3b49a65bf8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:43.685204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-fc6fe024-5085-4f8a-bfdc-3b49a65bf8a7 None None] MFA Rules not processed for user `da46b59a0cb5430589aeef1693b2ce50`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:43.686421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fc6fe024-5085-4f8a-bfdc-3b49a65bf8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:43.694225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1294/2605] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 22:56:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:56:43.989283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea0699-8239-4608-861e-a36bb3407cac service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:43.997780 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea0699-8239-4608-861e-a36bb3407cac service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:44.001361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d4e1e-b748-4ae3-8a8a-cc5f1025115c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:44.001997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d4e1e-b748-4ae3-8a8a-cc5f1025115c service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:44.002277 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d4e1e-b748-4ae3-8a8a-cc5f1025115c service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:44.002553 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d4e1e-b748-4ae3-8a8a-cc5f1025115c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:44.004500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-868d4e1e-b748-4ae3-8a8a-cc5f1025115c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:44.016017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d4e1e-b748-4ae3-8a8a-cc5f1025115c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:44.017747 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d4e1e-b748-4ae3-8a8a-cc5f1025115c service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:44.026369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1312/2606] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:44.351759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6fe024-5085-4f8a-bfdc-3b49a65bf8a7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:44.353169 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fc6fe024-5085-4f8a-bfdc-3b49a65bf8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:44.435227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6fe024-5085-4f8a-bfdc-3b49a65bf8a7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:44.440461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409e85a-ffec-4beb-89a2-30cd565ec697 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:44.441241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409e85a-ffec-4beb-89a2-30cd565ec697 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:44.441533 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409e85a-ffec-4beb-89a2-30cd565ec697 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:44.442049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409e85a-ffec-4beb-89a2-30cd565ec697 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:44.443939 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f409e85a-ffec-4beb-89a2-30cd565ec697 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:44.455920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409e85a-ffec-4beb-89a2-30cd565ec697 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:44.458046 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409e85a-ffec-4beb-89a2-30cd565ec697 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:44.468312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1295/2607] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:47.279347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d4e1e-b748-4ae3-8a8a-cc5f1025115c service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:47.280801 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-868d4e1e-b748-4ae3-8a8a-cc5f1025115c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:47.301616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d4e1e-b748-4ae3-8a8a-cc5f1025115c service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:47.311179 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd061c86-ecdf-43a8-b510-964ff3862ab4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:47.311896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd061c86-ecdf-43a8-b510-964ff3862ab4 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:47.312249 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd061c86-ecdf-43a8-b510-964ff3862ab4 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:47.312558 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd061c86-ecdf-43a8-b510-964ff3862ab4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:47.315191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-dd061c86-ecdf-43a8-b510-964ff3862ab4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:47.329524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd061c86-ecdf-43a8-b510-964ff3862ab4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:47.331128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd061c86-ecdf-43a8-b510-964ff3862ab4 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:47.353697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1313/2608] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:53.183278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409e85a-ffec-4beb-89a2-30cd565ec697 service placement] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:53.196737 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409e85a-ffec-4beb-89a2-30cd565ec697 service placement] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:53.202244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc7597-c9f7-42fb-b8e7-b795b7dda407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:53.202890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc7597-c9f7-42fb-b8e7-b795b7dda407 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:53.203171 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc7597-c9f7-42fb-b8e7-b795b7dda407 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:53.203448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc7597-c9f7-42fb-b8e7-b795b7dda407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:53.205200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f8bc7597-c9f7-42fb-b8e7-b795b7dda407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:53.217898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc7597-c9f7-42fb-b8e7-b795b7dda407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:53.219487 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc7597-c9f7-42fb-b8e7-b795b7dda407 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:53.230566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1296/2609] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:57.340413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=12 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:57:03.226454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=44 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:58:05.639098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd061c86-ecdf-43a8-b510-964ff3862ab4 service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:05.640510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-dd061c86-ecdf-43a8-b510-964ff3862ab4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:05.660190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd061c86-ecdf-43a8-b510-964ff3862ab4 service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:05.668190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e7fe6-9cba-4687-a866-b11280692cbb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:05.668887 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e7fe6-9cba-4687-a866-b11280692cbb service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:05.669183 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e7fe6-9cba-4687-a866-b11280692cbb service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:05.669504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e7fe6-9cba-4687-a866-b11280692cbb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:05.671279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-604e7fe6-9cba-4687-a866-b11280692cbb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:05.685490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e7fe6-9cba-4687-a866-b11280692cbb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:05.688546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e7fe6-9cba-4687-a866-b11280692cbb service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:05.702515 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1314/2610] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:58:14.977974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc7597-c9f7-42fb-b8e7-b795b7dda407 service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:14.979318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f8bc7597-c9f7-42fb-b8e7-b795b7dda407 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:15.006802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc7597-c9f7-42fb-b8e7-b795b7dda407 service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:15.012293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb9f62-0b93-4270-a66c-94adcd7621da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:15.012965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb9f62-0b93-4270-a66c-94adcd7621da service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:15.013249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb9f62-0b93-4270-a66c-94adcd7621da service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:15.013522 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb9f62-0b93-4270-a66c-94adcd7621da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:15.015274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bfbb9f62-0b93-4270-a66c-94adcd7621da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:15.085217 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb9f62-0b93-4270-a66c-94adcd7621da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:15.086818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb9f62-0b93-4270-a66c-94adcd7621da service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:15.104848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1297/2611] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:58:15.695221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=6 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:58:25.102240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=41 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:15.548123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e7fe6-9cba-4687-a866-b11280692cbb service placement] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:15.557390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e7fe6-9cba-4687-a866-b11280692cbb service placement] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:15.560990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cbc36-587f-469a-afc6-bc20d44d2a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:15.561737 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cbc36-587f-469a-afc6-bc20d44d2a82 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:15.562069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cbc36-587f-469a-afc6-bc20d44d2a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:15.562358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cbc36-587f-469a-afc6-bc20d44d2a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:15.571546 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cbc36-587f-469a-afc6-bc20d44d2a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:15.572953 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cbc36-587f-469a-afc6-bc20d44d2a82 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:15.584467 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1315/2612] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:15.837332 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb9f62-0b93-4270-a66c-94adcd7621da service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:15.847061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb9f62-0b93-4270-a66c-94adcd7621da service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:15.851062 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59726a-2735-4043-89d0-2178a0fc1f61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:15.851721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59726a-2735-4043-89d0-2178a0fc1f61 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:15.852008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59726a-2735-4043-89d0-2178a0fc1f61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:15.852423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59726a-2735-4043-89d0-2178a0fc1f61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:15.855724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-be59726a-2735-4043-89d0-2178a0fc1f61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-be59726a-2735-4043-89d0-2178a0fc1f61 service cinder] Could not find project: 671dbc43a5c04a26b23e278f3f7964bb.: keystone.exception.ProjectNotFound: Could not find project: 671dbc43a5c04a26b23e278f3f7964bb. Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 21 22:59:15.869883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:59:15.871296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:59:15.872687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:59:15.874009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:59:15.875383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 671dbc43a5c04a26b23e278f3f7964bb. Jul 21 22:59:15.875383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Jul 21 22:59:15.875383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1298/2613] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:15 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Jul 21 22:59:16.523176 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cbc36-587f-469a-afc6-bc20d44d2a82 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:16.532449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cbc36-587f-469a-afc6-bc20d44d2a82 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:16.536079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35daf3-9908-4624-957e-5efa9182cb53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:16.536754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35daf3-9908-4624-957e-5efa9182cb53 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:16.537043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35daf3-9908-4624-957e-5efa9182cb53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:16.537326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35daf3-9908-4624-957e-5efa9182cb53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:16.547513 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35daf3-9908-4624-957e-5efa9182cb53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:16.548977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35daf3-9908-4624-957e-5efa9182cb53 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:16.560041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1316/2614] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:17.028126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59726a-2735-4043-89d0-2178a0fc1f61 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:17.038222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59726a-2735-4043-89d0-2178a0fc1f61 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:17.041815 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8407b5-13cd-453a-bb6c-a1e5832c6450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:17.042449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8407b5-13cd-453a-bb6c-a1e5832c6450 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:17.042736 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8407b5-13cd-453a-bb6c-a1e5832c6450 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:17.043016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8407b5-13cd-453a-bb6c-a1e5832c6450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:17.054735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8407b5-13cd-453a-bb6c-a1e5832c6450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:17.056168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8407b5-13cd-453a-bb6c-a1e5832c6450 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:17.067159 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1299/2615] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:26.555181 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=12 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:27.062224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=10 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:16.222450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35daf3-9908-4624-957e-5efa9182cb53 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:16.234076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35daf3-9908-4624-957e-5efa9182cb53 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:16.240015 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcef779-cd66-4181-826c-02797d3219b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:16.240693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcef779-cd66-4181-826c-02797d3219b7 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:16.240983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcef779-cd66-4181-826c-02797d3219b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:16.241260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcef779-cd66-4181-826c-02797d3219b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:16.249134 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcef779-cd66-4181-826c-02797d3219b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:16.250703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcef779-cd66-4181-826c-02797d3219b7 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:16.260925 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1317/2616] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:00:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:00:26.256751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=6 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:17.027947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8407b5-13cd-453a-bb6c-a1e5832c6450 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:17.037009 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8407b5-13cd-453a-bb6c-a1e5832c6450 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:17.040657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381f3ba-5a3f-4f44-a4e9-72ef4d2ee517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:17.041355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381f3ba-5a3f-4f44-a4e9-72ef4d2ee517 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:17.041577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381f3ba-5a3f-4f44-a4e9-72ef4d2ee517 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:17.041850 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381f3ba-5a3f-4f44-a4e9-72ef4d2ee517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:17.049523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381f3ba-5a3f-4f44-a4e9-72ef4d2ee517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:17.051532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381f3ba-5a3f-4f44-a4e9-72ef4d2ee517 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:17.059999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1300/2617] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:27.056176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=6 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:44.610871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcef779-cd66-4181-826c-02797d3219b7 service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:44.639645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcef779-cd66-4181-826c-02797d3219b7 service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:44.645220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ab2ea-f6f6-436a-8dd8-4428cc38416d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:44.645865 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ab2ea-f6f6-436a-8dd8-4428cc38416d service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:44.646175 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ab2ea-f6f6-436a-8dd8-4428cc38416d service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:44.646969 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ab2ea-f6f6-436a-8dd8-4428cc38416d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:44.656436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ab2ea-f6f6-436a-8dd8-4428cc38416d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:44.658032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ab2ea-f6f6-436a-8dd8-4428cc38416d service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:44.669144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1318/2618] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:54.664818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=13 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:03:10.612173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381f3ba-5a3f-4f44-a4e9-72ef4d2ee517 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:10.655805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381f3ba-5a3f-4f44-a4e9-72ef4d2ee517 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:10.660904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265afa4-335b-4cf9-aee9-7c03b4fe5902 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:10.661541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265afa4-335b-4cf9-aee9-7c03b4fe5902 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:10.661828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265afa4-335b-4cf9-aee9-7c03b4fe5902 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:10.662108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265afa4-335b-4cf9-aee9-7c03b4fe5902 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:10.671477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265afa4-335b-4cf9-aee9-7c03b4fe5902 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:10.673135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265afa4-335b-4cf9-aee9-7c03b4fe5902 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:10.683769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1301/2619] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:03:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:03:16.226128 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ab2ea-f6f6-436a-8dd8-4428cc38416d service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:16.263640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ab2ea-f6f6-436a-8dd8-4428cc38416d service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:16.267316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe6030-c887-4665-a47c-bb7f77b654ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:16.267966 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe6030-c887-4665-a47c-bb7f77b654ea service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:16.268252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe6030-c887-4665-a47c-bb7f77b654ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:16.269333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe6030-c887-4665-a47c-bb7f77b654ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:16.278848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe6030-c887-4665-a47c-bb7f77b654ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:16.280285 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe6030-c887-4665-a47c-bb7f77b654ea service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:16.290589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1319/2620] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:03:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:03:20.679474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=19 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:03:26.285201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=18 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:04:43.919698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265afa4-335b-4cf9-aee9-7c03b4fe5902 service placement] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:04:43.921150 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4265afa4-335b-4cf9-aee9-7c03b4fe5902 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:43.964687 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265afa4-335b-4cf9-aee9-7c03b4fe5902 service placement] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:04:43.968619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f67ba-a3d0-42dc-8ed6-e6ff01fdcaac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:04:43.969261 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f67ba-a3d0-42dc-8ed6-e6ff01fdcaac service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:04:43.969543 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f67ba-a3d0-42dc-8ed6-e6ff01fdcaac service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:04:43.969818 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f67ba-a3d0-42dc-8ed6-e6ff01fdcaac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:04:43.971691 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8f7f67ba-a3d0-42dc-8ed6-e6ff01fdcaac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:44.052340 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f67ba-a3d0-42dc-8ed6-e6ff01fdcaac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:04:44.053852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f67ba-a3d0-42dc-8ed6-e6ff01fdcaac service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:04:44.073705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1302/2621] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:04:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:04:54.071057 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=55 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:05:16.538369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe6030-c887-4665-a47c-bb7f77b654ea service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:05:16.539944 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-affe6030-c887-4665-a47c-bb7f77b654ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:05:16.602117 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe6030-c887-4665-a47c-bb7f77b654ea service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:05:16.606411 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7653f71-1eab-4a5a-9915-906614e1aeab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:05:16.607065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7653f71-1eab-4a5a-9915-906614e1aeab service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:16.607348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7653f71-1eab-4a5a-9915-906614e1aeab service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:16.607627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7653f71-1eab-4a5a-9915-906614e1aeab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:16.609632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d7653f71-1eab-4a5a-9915-906614e1aeab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:05:16.631219 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f67ba-a3d0-42dc-8ed6-e6ff01fdcaac service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:05:16.632826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8f7f67ba-a3d0-42dc-8ed6-e6ff01fdcaac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:05:16.696230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7653f71-1eab-4a5a-9915-906614e1aeab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:05:16.698139 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7653f71-1eab-4a5a-9915-906614e1aeab service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:05:16.711884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f67ba-a3d0-42dc-8ed6-e6ff01fdcaac service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:05:16.716281 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63a90d-c81d-4aaa-85ad-be0f1596ae7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:05:16.716958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63a90d-c81d-4aaa-85ad-be0f1596ae7a service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:16.717241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63a90d-c81d-4aaa-85ad-be0f1596ae7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:16.717475 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1320/2622] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:05:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:05:16.717619 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63a90d-c81d-4aaa-85ad-be0f1596ae7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:16.719292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-af63a90d-c81d-4aaa-85ad-be0f1596ae7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:05:16.806370 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63a90d-c81d-4aaa-85ad-be0f1596ae7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:05:16.807803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63a90d-c81d-4aaa-85ad-be0f1596ae7a service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:05:16.827400 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1303/2623] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:05:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:05:19.887402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7653f71-1eab-4a5a-9915-906614e1aeab service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:05:19.898782 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7653f71-1eab-4a5a-9915-906614e1aeab service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:05:19.903766 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36418acd-03c2-4bd4-b7de-30907d0a847a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:05:19.904438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36418acd-03c2-4bd4-b7de-30907d0a847a service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:19.904754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36418acd-03c2-4bd4-b7de-30907d0a847a service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:19.905034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36418acd-03c2-4bd4-b7de-30907d0a847a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:19.906787 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-36418acd-03c2-4bd4-b7de-30907d0a847a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:05:19.934112 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36418acd-03c2-4bd4-b7de-30907d0a847a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:05:19.935853 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36418acd-03c2-4bd4-b7de-30907d0a847a service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:05:19.949702 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1321/2624] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:05:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:05:19.951534 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63a90d-c81d-4aaa-85ad-be0f1596ae7a service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:05:19.953093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-af63a90d-c81d-4aaa-85ad-be0f1596ae7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:05:19.966838 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63a90d-c81d-4aaa-85ad-be0f1596ae7a service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:05:19.971840 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267a900-efd6-4ba0-97cf-72f675bb620c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:05:19.972830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267a900-efd6-4ba0-97cf-72f675bb620c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:19.973183 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267a900-efd6-4ba0-97cf-72f675bb620c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:19.973531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267a900-efd6-4ba0-97cf-72f675bb620c admin admin] PATH_INFO: `/v3/users/da46b59a0cb5430589aeef1693b2ce50` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:19.974816 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267a900-efd6-4ba0-97cf-72f675bb620c admin admin] RBAC: Authorizing `identity:delete_user(user_id=da46b59a0cb5430589aeef1693b2ce50)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:05:19.976883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267a900-efd6-4ba0-97cf-72f675bb620c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:05:20.024563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e267a900-efd6-4ba0-97cf-72f675bb620c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da46b59a0cb5430589aeef1693b2ce50'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:05:20.031720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e267a900-efd6-4ba0-97cf-72f675bb620c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da46b59a0cb5430589aeef1693b2ce50'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:05:20.032886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:05:20.046833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e267a900-efd6-4ba0-97cf-72f675bb620c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da46b59a0cb5430589aeef1693b2ce50'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:05:20.054063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e267a900-efd6-4ba0-97cf-72f675bb620c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da46b59a0cb5430589aeef1693b2ce50'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:05:20.054696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e267a900-efd6-4ba0-97cf-72f675bb620c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da46b59a0cb5430589aeef1693b2ce50'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:05:20.060841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1304/2625] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:05:19 2022] DELETE /identity/v3/users/da46b59a0cb5430589aeef1693b2ce50 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:05:20.234659 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36418acd-03c2-4bd4-b7de-30907d0a847a service placement] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:05:20.251260 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36418acd-03c2-4bd4-b7de-30907d0a847a service placement] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:05:20.309174 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3686e0-984e-4b5b-b525-8665edeaefa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:05:20.310043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3686e0-984e-4b5b-b525-8665edeaefa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:20.310330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3686e0-984e-4b5b-b525-8665edeaefa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:20.310608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3686e0-984e-4b5b-b525-8665edeaefa4 admin admin] PATH_INFO: `/v3/projects/613f21f4b5164e6087e734f9b447e961` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:20.311848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3686e0-984e-4b5b-b525-8665edeaefa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=613f21f4b5164e6087e734f9b447e961)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:05:20.313722 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3686e0-984e-4b5b-b525-8665edeaefa4 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:05:20.315924 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:05:20.360942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9c3686e0-984e-4b5b-b525-8665edeaefa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '613f21f4b5164e6087e734f9b447e961', 'request_id': 'req-9c3686e0-984e-4b5b-b525-8665edeaefa4'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:05:20.367841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9c3686e0-984e-4b5b-b525-8665edeaefa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '613f21f4b5164e6087e734f9b447e961', 'request_id': 'req-9c3686e0-984e-4b5b-b525-8665edeaefa4'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:05:20.373963 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9c3686e0-984e-4b5b-b525-8665edeaefa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '613f21f4b5164e6087e734f9b447e961', 'request_id': 'req-9c3686e0-984e-4b5b-b525-8665edeaefa4'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:05:20.384996 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9c3686e0-984e-4b5b-b525-8665edeaefa4 admin admin] The token cache is being invalidate because project 613f21f4b5164e6087e734f9b447e961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:05:20.385286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9c3686e0-984e-4b5b-b525-8665edeaefa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:05:20.385947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9c3686e0-984e-4b5b-b525-8665edeaefa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:05:20.387647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1322/2626] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:05:20 2022] DELETE /identity/v3/projects/613f21f4b5164e6087e734f9b447e961 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:05:20.393383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfeb799-34cb-4a76-9f33-9391a9f7e40b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:20.393675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfeb799-34cb-4a76-9f33-9391a9f7e40b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:20.393958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfeb799-34cb-4a76-9f33-9391a9f7e40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:20.450150 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9cfeb799-34cb-4a76-9f33-9391a9f7e40b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:05:20.451359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9cfeb799-34cb-4a76-9f33-9391a9f7e40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:05:20.509195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1305/2627] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:05:20.516889 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebea9a-9695-4532-b3f2-bfc856f85527 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:20.517251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebea9a-9695-4532-b3f2-bfc856f85527 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:20.517618 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebea9a-9695-4532-b3f2-bfc856f85527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:20.535601 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-4eebea9a-9695-4532-b3f2-bfc856f85527 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:05:20.536824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4eebea9a-9695-4532-b3f2-bfc856f85527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:05:20.545636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1323/2628] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:05:20.549657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfeb799-34cb-4a76-9f33-9391a9f7e40b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:05:20.559680 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfeb799-34cb-4a76-9f33-9391a9f7e40b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:05:20.564252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfce2f-85ea-480c-821c-78164530d0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:05:20.564874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfce2f-85ea-480c-821c-78164530d0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:20.565155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfce2f-85ea-480c-821c-78164530d0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:20.565428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfce2f-85ea-480c-821c-78164530d0c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:20.566144 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cfce2f-85ea-480c-821c-78164530d0c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:05:20.566446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cfce2f-85ea-480c-821c-78164530d0c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:05:20.568038 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cfce2f-85ea-480c-821c-78164530d0c0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:05:20.579481 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1306/2629] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 23:05:20.583681 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebea9a-9695-4532-b3f2-bfc856f85527 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:05:20.593703 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebea9a-9695-4532-b3f2-bfc856f85527 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:05:20.598436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122a30a-267a-4ac8-bbbc-80de75c44b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:05:20.599169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122a30a-267a-4ac8-bbbc-80de75c44b41 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:20.599458 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122a30a-267a-4ac8-bbbc-80de75c44b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:20.599767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122a30a-267a-4ac8-bbbc-80de75c44b41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:20.600503 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122a30a-267a-4ac8-bbbc-80de75c44b41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:05:20.602398 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122a30a-267a-4ac8-bbbc-80de75c44b41 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:05:20.638220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1324/2630] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:05:20 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:05:20.642246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfce2f-85ea-480c-821c-78164530d0c0 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:05:20.652992 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfce2f-85ea-480c-821c-78164530d0c0 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:05:20.707964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8f890-7a71-4849-b398-024e46e9899e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:05:20.708649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8f890-7a71-4849-b398-024e46e9899e admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:20.708936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8f890-7a71-4849-b398-024e46e9899e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:20.709209 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8f890-7a71-4849-b398-024e46e9899e admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:20.709882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8f890-7a71-4849-b398-024e46e9899e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:05:20.711775 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8f890-7a71-4849-b398-024e46e9899e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:05:20.732613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1307/2631] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:05:20 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:05:20.736860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122a30a-267a-4ac8-bbbc-80de75c44b41 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:05:20.747319 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122a30a-267a-4ac8-bbbc-80de75c44b41 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:05:20.752386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f6fa2-7c2b-4584-be8b-21c534f3957f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:05:20.753083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6fa2-7c2b-4584-be8b-21c534f3957f admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:20.753395 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6fa2-7c2b-4584-be8b-21c534f3957f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:20.753671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6fa2-7c2b-4584-be8b-21c534f3957f admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:20.754396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f6fa2-7c2b-4584-be8b-21c534f3957f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:05:20.755970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f6fa2-7c2b-4584-be8b-21c534f3957f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:05:20.765834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1325/2632] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:05:20.769640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8f890-7a71-4849-b398-024e46e9899e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:05:20.780106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8f890-7a71-4849-b398-024e46e9899e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:05:20.784769 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb18385-0f55-4a49-afc1-1715def0f4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:05:20.785417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb18385-0f55-4a49-afc1-1715def0f4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:20.785699 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb18385-0f55-4a49-afc1-1715def0f4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:20.785975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb18385-0f55-4a49-afc1-1715def0f4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:20.786693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb18385-0f55-4a49-afc1-1715def0f4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:05:20.788271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb18385-0f55-4a49-afc1-1715def0f4d3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:05:20.799151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1308/2633] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:05:20.803228 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f6fa2-7c2b-4584-be8b-21c534f3957f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:05:20.815158 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f6fa2-7c2b-4584-be8b-21c534f3957f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:05:20.819696 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c69eac-e5ca-4da3-aab0-3f50f219948d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:05:20.820634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c69eac-e5ca-4da3-aab0-3f50f219948d admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:20.820913 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c69eac-e5ca-4da3-aab0-3f50f219948d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:20.821206 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c69eac-e5ca-4da3-aab0-3f50f219948d admin admin] PATH_INFO: `/v3/projects/1c53da2fff85471fb98fa44fa41a7bcd/users/2e9e0b67144442c7aaf11dbe2c42a344/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:20.836166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c69eac-e5ca-4da3-aab0-3f50f219948d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c53da2fff85471fb98fa44fa41a7bcd, user_id=2e9e0b67144442c7aaf11dbe2c42a344, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:05:20.838847 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c69eac-e5ca-4da3-aab0-3f50f219948d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:05:20.851293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1326/2634] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:05:20 2022] PUT /identity/v3/projects/1c53da2fff85471fb98fa44fa41a7bcd/users/2e9e0b67144442c7aaf11dbe2c42a344/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:05:20.861870 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf89960-67b7-4cf1-82e1-c5096e2347d0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:20.862164 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf89960-67b7-4cf1-82e1-c5096e2347d0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:20.862443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf89960-67b7-4cf1-82e1-c5096e2347d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:20.881602 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:05:20.892962 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2cf89960-67b7-4cf1-82e1-c5096e2347d0 None None] MFA Rules not processed for user `2e9e0b67144442c7aaf11dbe2c42a344`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:05:20.894419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2cf89960-67b7-4cf1-82e1-c5096e2347d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:05:20.956918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1309/2635] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:05:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:05:20.968950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c69eac-e5ca-4da3-aab0-3f50f219948d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:05:20.970656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-88c69eac-e5ca-4da3-aab0-3f50f219948d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:05:21.043299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c69eac-e5ca-4da3-aab0-3f50f219948d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:05:21.048816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebefc3-d061-4d1b-95a2-47647b5803ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:05:21.049459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebefc3-d061-4d1b-95a2-47647b5803ec service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:05:21.049744 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebefc3-d061-4d1b-95a2-47647b5803ec service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:05:21.050021 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebefc3-d061-4d1b-95a2-47647b5803ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:05:21.058437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebefc3-d061-4d1b-95a2-47647b5803ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:05:21.060069 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebefc3-d061-4d1b-95a2-47647b5803ec service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:05:21.069039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1327/2636] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:05:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:05:30.954040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=28 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:05:31.064898 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:06:44.382117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf89960-67b7-4cf1-82e1-c5096e2347d0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:06:44.383530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2cf89960-67b7-4cf1-82e1-c5096e2347d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:06:44.447767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf89960-67b7-4cf1-82e1-c5096e2347d0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:06:44.451378 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2237a-17d0-4ac4-813c-a07250014dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:06:44.452026 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2237a-17d0-4ac4-813c-a07250014dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:06:44.452314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2237a-17d0-4ac4-813c-a07250014dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:06:44.452633 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2237a-17d0-4ac4-813c-a07250014dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:06:44.454425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e8d2237a-17d0-4ac4-813c-a07250014dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:06:44.468153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2237a-17d0-4ac4-813c-a07250014dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:06:44.469623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2237a-17d0-4ac4-813c-a07250014dc5 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:06:44.481117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1310/2637] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:06:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:06:44.579690 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebefc3-d061-4d1b-95a2-47647b5803ec service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:06:44.589422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebefc3-d061-4d1b-95a2-47647b5803ec service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:06:44.592958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede56038-b4d8-4713-b64c-01cc527ce7b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:06:44.593607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede56038-b4d8-4713-b64c-01cc527ce7b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:06:44.593895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede56038-b4d8-4713-b64c-01cc527ce7b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:06:44.594175 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede56038-b4d8-4713-b64c-01cc527ce7b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:06:44.595947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ede56038-b4d8-4713-b64c-01cc527ce7b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:06:44.609984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede56038-b4d8-4713-b64c-01cc527ce7b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:06:44.611405 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede56038-b4d8-4713-b64c-01cc527ce7b4 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:06:44.622480 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1328/2638] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:06:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:06:54.475673 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=26 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:06:54.617444 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=6 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:07:12.220828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2237a-17d0-4ac4-813c-a07250014dc5 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:07:12.222594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e8d2237a-17d0-4ac4-813c-a07250014dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:07:12.280353 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2237a-17d0-4ac4-813c-a07250014dc5 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:07:12.284008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900c189-1d00-4a94-b870-bf670d18294a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:07:12.284709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900c189-1d00-4a94-b870-bf670d18294a service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:07:12.284995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900c189-1d00-4a94-b870-bf670d18294a service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:07:12.285273 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900c189-1d00-4a94-b870-bf670d18294a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:07:12.287043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2900c189-1d00-4a94-b870-bf670d18294a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:07:12.296714 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900c189-1d00-4a94-b870-bf670d18294a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:07:12.298147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900c189-1d00-4a94-b870-bf670d18294a service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:07:12.306176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1311/2639] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:07:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:07:22.302504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=26 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:08:13.609539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede56038-b4d8-4713-b64c-01cc527ce7b4 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:13.611085 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ede56038-b4d8-4713-b64c-01cc527ce7b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:13.685242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede56038-b4d8-4713-b64c-01cc527ce7b4 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:13.690498 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e17faa-bd0c-4bd0-9a5c-c5333300036f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:13.691152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e17faa-bd0c-4bd0-9a5c-c5333300036f service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:13.691435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e17faa-bd0c-4bd0-9a5c-c5333300036f service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:13.691712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e17faa-bd0c-4bd0-9a5c-c5333300036f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:13.693524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c6e17faa-bd0c-4bd0-9a5c-c5333300036f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:13.707293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e17faa-bd0c-4bd0-9a5c-c5333300036f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:13.708983 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e17faa-bd0c-4bd0-9a5c-c5333300036f service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:13.720733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1329/2640] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:08:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:08:23.715346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=32 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:08:44.698664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900c189-1d00-4a94-b870-bf670d18294a service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:44.706536 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900c189-1d00-4a94-b870-bf670d18294a service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:44.710246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a98110-b1a2-4fb7-a76e-0ce4b2433b05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:44.710908 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a98110-b1a2-4fb7-a76e-0ce4b2433b05 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:44.711188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a98110-b1a2-4fb7-a76e-0ce4b2433b05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:44.711465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a98110-b1a2-4fb7-a76e-0ce4b2433b05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:44.713967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-40a98110-b1a2-4fb7-a76e-0ce4b2433b05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:44.792413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a98110-b1a2-4fb7-a76e-0ce4b2433b05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:44.793869 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a98110-b1a2-4fb7-a76e-0ce4b2433b05 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:44.812019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1312/2641] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:08:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:08:44.927349 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e17faa-bd0c-4bd0-9a5c-c5333300036f service placement] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:45.011062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e17faa-bd0c-4bd0-9a5c-c5333300036f service placement] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:45.015647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:45.016542 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:45.016830 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:45.017111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] PATH_INFO: `/v3/users/2e9e0b67144442c7aaf11dbe2c42a344` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:45.018312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e9e0b67144442c7aaf11dbe2c42a344)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:45.020214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:45.066452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e9e0b67144442c7aaf11dbe2c42a344'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:45.073036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e9e0b67144442c7aaf11dbe2c42a344'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:45.073812 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e9e0b67144442c7aaf11dbe2c42a344'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:45.080624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e9e0b67144442c7aaf11dbe2c42a344'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:45.081250 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e9e0b67144442c7aaf11dbe2c42a344'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:45.098374 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1330/2642] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:08:44 2022] DELETE /identity/v3/users/2e9e0b67144442c7aaf11dbe2c42a344 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:08:45.102819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7bf8e-9ae2-4c5a-9bfe-be29a93a0b21 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:45.103107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7bf8e-9ae2-4c5a-9bfe-be29a93a0b21 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:45.103380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7bf8e-9ae2-4c5a-9bfe-be29a93a0b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:45.121137 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a0d7bf8e-9ae2-4c5a-9bfe-be29a93a0b21 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:08:45.122344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a0d7bf8e-9ae2-4c5a-9bfe-be29a93a0b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:45.179885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1313/2643] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:08:45.189492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:45.190936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:45.249075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655553-ad3d-448f-af5e-95e8182c94ea admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:45.252718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037e95c-af85-42be-adb7-a0d975141c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:45.253367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037e95c-af85-42be-adb7-a0d975141c2f service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:45.253654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037e95c-af85-42be-adb7-a0d975141c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:45.253936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037e95c-af85-42be-adb7-a0d975141c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:45.263622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037e95c-af85-42be-adb7-a0d975141c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:45.265088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037e95c-af85-42be-adb7-a0d975141c2f service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:45.274559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1331/2644] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:08:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:08:45.553059 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7bf8e-9ae2-4c5a-9bfe-be29a93a0b21 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:45.554534 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a0d7bf8e-9ae2-4c5a-9bfe-be29a93a0b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:45.567328 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7bf8e-9ae2-4c5a-9bfe-be29a93a0b21 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:45.571862 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f62f3-ee08-4581-893b-3441c0980f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:45.573416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f62f3-ee08-4581-893b-3441c0980f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:45.573700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f62f3-ee08-4581-893b-3441c0980f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:45.573970 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f62f3-ee08-4581-893b-3441c0980f5f admin admin] PATH_INFO: `/v3/projects/1c53da2fff85471fb98fa44fa41a7bcd` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:45.575181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f62f3-ee08-4581-893b-3441c0980f5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c53da2fff85471fb98fa44fa41a7bcd)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:45.577139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f62f3-ee08-4581-893b-3441c0980f5f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:45.625130 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8b6f62f3-ee08-4581-893b-3441c0980f5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c53da2fff85471fb98fa44fa41a7bcd', 'request_id': 'req-8b6f62f3-ee08-4581-893b-3441c0980f5f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:45.630502 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8b6f62f3-ee08-4581-893b-3441c0980f5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c53da2fff85471fb98fa44fa41a7bcd', 'request_id': 'req-8b6f62f3-ee08-4581-893b-3441c0980f5f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:45.638845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8b6f62f3-ee08-4581-893b-3441c0980f5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c53da2fff85471fb98fa44fa41a7bcd', 'request_id': 'req-8b6f62f3-ee08-4581-893b-3441c0980f5f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:45.639966 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:08:45.652796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8b6f62f3-ee08-4581-893b-3441c0980f5f admin admin] The token cache is being invalidate because project 1c53da2fff85471fb98fa44fa41a7bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:08:45.653086 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8b6f62f3-ee08-4581-893b-3441c0980f5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:45.653729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8b6f62f3-ee08-4581-893b-3441c0980f5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:45.655279 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1314/2645] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:08:45 2022] DELETE /identity/v3/projects/1c53da2fff85471fb98fa44fa41a7bcd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:08:45.660119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e241e-f49b-45f4-86b9-4cd79be2e147 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:45.660420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e241e-f49b-45f4-86b9-4cd79be2e147 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:45.660778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e241e-f49b-45f4-86b9-4cd79be2e147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:45.678070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-e67e241e-f49b-45f4-86b9-4cd79be2e147 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:08:45.679304 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-e67e241e-f49b-45f4-86b9-4cd79be2e147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:45.696820 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:08:45.742028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1332/2646] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:08:45.751705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b73ae3-1cda-4df7-882e-d9ebe3b620fb None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:45.752207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b73ae3-1cda-4df7-882e-d9ebe3b620fb None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:45.752321 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b73ae3-1cda-4df7-882e-d9ebe3b620fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:45.770925 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-83b73ae3-1cda-4df7-882e-d9ebe3b620fb None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:08:45.772195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-83b73ae3-1cda-4df7-882e-d9ebe3b620fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:45.780314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1315/2647] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:08:45.784434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e241e-f49b-45f4-86b9-4cd79be2e147 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:45.794324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e241e-f49b-45f4-86b9-4cd79be2e147 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:45.798900 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7943d1-4e14-44e0-93e2-33f2bd3fc908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:45.799493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7943d1-4e14-44e0-93e2-33f2bd3fc908 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:45.799776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7943d1-4e14-44e0-93e2-33f2bd3fc908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:45.800058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7943d1-4e14-44e0-93e2-33f2bd3fc908 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:45.800825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7943d1-4e14-44e0-93e2-33f2bd3fc908 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:08:45.801132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7943d1-4e14-44e0-93e2-33f2bd3fc908 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:45.802715 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7943d1-4e14-44e0-93e2-33f2bd3fc908 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:45.814944 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1333/2648] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:08:45.819032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b73ae3-1cda-4df7-882e-d9ebe3b620fb None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:45.828911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b73ae3-1cda-4df7-882e-d9ebe3b620fb None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:45.833574 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4a4c4-4ebf-4986-bf51-4cb3ce54a82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:45.834218 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4a4c4-4ebf-4986-bf51-4cb3ce54a82e admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:45.834503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4a4c4-4ebf-4986-bf51-4cb3ce54a82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:45.834781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4a4c4-4ebf-4986-bf51-4cb3ce54a82e admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:45.835474 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4a4c4-4ebf-4986-bf51-4cb3ce54a82e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:45.837422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4a4c4-4ebf-4986-bf51-4cb3ce54a82e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:45.862441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1316/2649] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:08:45 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:08:45.866473 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7943d1-4e14-44e0-93e2-33f2bd3fc908 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:45.877405 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7943d1-4e14-44e0-93e2-33f2bd3fc908 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:45.933469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388d694-3e45-4279-a2d0-12cf870ad195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:45.934129 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388d694-3e45-4279-a2d0-12cf870ad195 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:45.934464 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388d694-3e45-4279-a2d0-12cf870ad195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:45.934718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388d694-3e45-4279-a2d0-12cf870ad195 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:45.935404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388d694-3e45-4279-a2d0-12cf870ad195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:45.937371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388d694-3e45-4279-a2d0-12cf870ad195 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:45.957749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1334/2650] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:08:45 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:08:45.961907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4a4c4-4ebf-4986-bf51-4cb3ce54a82e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:45.972438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4a4c4-4ebf-4986-bf51-4cb3ce54a82e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:45.977067 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893653f-ec5d-4cdc-b018-d03f726fc301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:45.977722 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893653f-ec5d-4cdc-b018-d03f726fc301 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:45.978007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893653f-ec5d-4cdc-b018-d03f726fc301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:45.978285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893653f-ec5d-4cdc-b018-d03f726fc301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:45.979135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893653f-ec5d-4cdc-b018-d03f726fc301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:45.980759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893653f-ec5d-4cdc-b018-d03f726fc301 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:45.990971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1317/2651] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:08:45.995905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388d694-3e45-4279-a2d0-12cf870ad195 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:46.005846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388d694-3e45-4279-a2d0-12cf870ad195 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:46.010967 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b923455-1131-4549-a817-14bba38f248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:46.011617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b923455-1131-4549-a817-14bba38f248f admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:46.011901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b923455-1131-4549-a817-14bba38f248f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:46.012174 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b923455-1131-4549-a817-14bba38f248f admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:46.012930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b923455-1131-4549-a817-14bba38f248f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:46.014555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b923455-1131-4549-a817-14bba38f248f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:46.024217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1335/2652] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:08:46.028520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893653f-ec5d-4cdc-b018-d03f726fc301 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:46.038384 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893653f-ec5d-4cdc-b018-d03f726fc301 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:46.043761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b94ba6-9ea2-44c0-b15b-cb36f67a9dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:46.044675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b94ba6-9ea2-44c0-b15b-cb36f67a9dba admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:46.044958 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b94ba6-9ea2-44c0-b15b-cb36f67a9dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:46.045234 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b94ba6-9ea2-44c0-b15b-cb36f67a9dba admin admin] PATH_INFO: `/v3/projects/f80278af085f42c5b2254e60312444f0/users/f6d8aab1367a436a9a6f085f107f59ba/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:46.060131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b94ba6-9ea2-44c0-b15b-cb36f67a9dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f80278af085f42c5b2254e60312444f0, user_id=f6d8aab1367a436a9a6f085f107f59ba, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:46.062746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b94ba6-9ea2-44c0-b15b-cb36f67a9dba admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:46.075113 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1318/2653] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:08:46 2022] PUT /identity/v3/projects/f80278af085f42c5b2254e60312444f0/users/f6d8aab1367a436a9a6f085f107f59ba/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:08:46.085294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7bce1-ac08-4643-95b6-ce8aee6f9f33 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:46.085588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7bce1-ac08-4643-95b6-ce8aee6f9f33 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:46.085865 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7bce1-ac08-4643-95b6-ce8aee6f9f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:46.112022 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-d4b7bce1-ac08-4643-95b6-ce8aee6f9f33 None None] MFA Rules not processed for user `f6d8aab1367a436a9a6f085f107f59ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:08:46.113254 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d4b7bce1-ac08-4643-95b6-ce8aee6f9f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:46.177149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1336/2654] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:08:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:08:46.181675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b94ba6-9ea2-44c0-b15b-cb36f67a9dba admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:46.243006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b94ba6-9ea2-44c0-b15b-cb36f67a9dba admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:46.248690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec423c10-e796-44c2-b117-b5a345bd6136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:46.249377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec423c10-e796-44c2-b117-b5a345bd6136 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:46.249659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec423c10-e796-44c2-b117-b5a345bd6136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:46.249932 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec423c10-e796-44c2-b117-b5a345bd6136 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:46.250629 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec423c10-e796-44c2-b117-b5a345bd6136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:46.252560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec423c10-e796-44c2-b117-b5a345bd6136 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:46.276510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1319/2655] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:08:46 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:08:46.280639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7bce1-ac08-4643-95b6-ce8aee6f9f33 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:46.292359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7bce1-ac08-4643-95b6-ce8aee6f9f33 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:46.304625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:08:46.363163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cfd6b-1842-4323-a907-e5efa20111c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:46.363758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cfd6b-1842-4323-a907-e5efa20111c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:46.364046 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cfd6b-1842-4323-a907-e5efa20111c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:46.364330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cfd6b-1842-4323-a907-e5efa20111c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:46.365084 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cfd6b-1842-4323-a907-e5efa20111c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:46.366993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cfd6b-1842-4323-a907-e5efa20111c9 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:46.388121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1337/2656] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:08:46 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:08:46.392212 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec423c10-e796-44c2-b117-b5a345bd6136 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:46.402758 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec423c10-e796-44c2-b117-b5a345bd6136 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:46.407358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509022e-a140-455c-8806-d9ea50bc5847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:46.408011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509022e-a140-455c-8806-d9ea50bc5847 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:46.408295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509022e-a140-455c-8806-d9ea50bc5847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:46.408608 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509022e-a140-455c-8806-d9ea50bc5847 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:46.409331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509022e-a140-455c-8806-d9ea50bc5847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:46.410916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509022e-a140-455c-8806-d9ea50bc5847 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:46.422558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1320/2657] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:08:46.426677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cfd6b-1842-4323-a907-e5efa20111c9 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:46.436998 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cfd6b-1842-4323-a907-e5efa20111c9 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:46.441559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8786d5b-76d3-4b7e-885f-406af6e4d640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:46.442445 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8786d5b-76d3-4b7e-885f-406af6e4d640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:46.442727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8786d5b-76d3-4b7e-885f-406af6e4d640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:46.443001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8786d5b-76d3-4b7e-885f-406af6e4d640 admin admin] PATH_INFO: `/v3/projects/85922b61cf2843b99bcec7e1b3bc545b/users/ac5bbd1cc0c74c7296470651204657e6/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:46.457436 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8786d5b-76d3-4b7e-885f-406af6e4d640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85922b61cf2843b99bcec7e1b3bc545b, user_id=ac5bbd1cc0c74c7296470651204657e6, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:46.460050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8786d5b-76d3-4b7e-885f-406af6e4d640 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:46.473089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1338/2658] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:08:46 2022] PUT /identity/v3/projects/85922b61cf2843b99bcec7e1b3bc545b/users/ac5bbd1cc0c74c7296470651204657e6/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:08:46.476975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509022e-a140-455c-8806-d9ea50bc5847 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:46.487519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509022e-a140-455c-8806-d9ea50bc5847 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:46.543658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114e58b-a076-4d9c-a9ff-0447e053d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:46.544326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e58b-a076-4d9c-a9ff-0447e053d14a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:46.544642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e58b-a076-4d9c-a9ff-0447e053d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:46.544920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e58b-a076-4d9c-a9ff-0447e053d14a admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:46.545723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114e58b-a076-4d9c-a9ff-0447e053d14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:46.547315 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114e58b-a076-4d9c-a9ff-0447e053d14a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:46.552381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:08:46.557635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1321/2659] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:08:46.561876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8786d5b-76d3-4b7e-885f-406af6e4d640 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:46.571976 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8786d5b-76d3-4b7e-885f-406af6e4d640 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:46.577006 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888af7c-0538-45f1-9c6b-86dd3272f612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:46.577874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888af7c-0538-45f1-9c6b-86dd3272f612 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:46.578155 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888af7c-0538-45f1-9c6b-86dd3272f612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:46.578430 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888af7c-0538-45f1-9c6b-86dd3272f612 admin admin] PATH_INFO: `/v3/projects/85922b61cf2843b99bcec7e1b3bc545b/users/ac5bbd1cc0c74c7296470651204657e6/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:46.580392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888af7c-0538-45f1-9c6b-86dd3272f612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85922b61cf2843b99bcec7e1b3bc545b, user_id=ac5bbd1cc0c74c7296470651204657e6, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:46.583025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888af7c-0538-45f1-9c6b-86dd3272f612 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:46.593742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1339/2660] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:08:46 2022] PUT /identity/v3/projects/85922b61cf2843b99bcec7e1b3bc545b/users/ac5bbd1cc0c74c7296470651204657e6/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:08:46.603721 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bfcb2-48ed-47d5-8aa9-575107225e6f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:46.604547 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bfcb2-48ed-47d5-8aa9-575107225e6f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:46.604834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bfcb2-48ed-47d5-8aa9-575107225e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:46.630953 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-381bfcb2-48ed-47d5-8aa9-575107225e6f None None] MFA Rules not processed for user `ac5bbd1cc0c74c7296470651204657e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:08:46.632125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-381bfcb2-48ed-47d5-8aa9-575107225e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:46.701808 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1322/2661] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:08:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:08:56.590033 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=36,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:08:56.698799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=34 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:14.968740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888af7c-0538-45f1-9c6b-86dd3272f612 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:14.970781 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7888af7c-0538-45f1-9c6b-86dd3272f612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:15.034276 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bfcb2-48ed-47d5-8aa9-575107225e6f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:15.036456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-381bfcb2-48ed-47d5-8aa9-575107225e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:15.057592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888af7c-0538-45f1-9c6b-86dd3272f612 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:15.061366 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9955eb-f08e-415c-bac9-67cf0c337521 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b67db080874f3ebb508239c7a8ccf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:15.062033 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9955eb-f08e-415c-bac9-67cf0c337521 service glance] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:15.062318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9955eb-f08e-415c-bac9-67cf0c337521 service glance] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:15.062593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9955eb-f08e-415c-bac9-67cf0c337521 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:15.064600 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3e9955eb-f08e-415c-bac9-67cf0c337521 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application [None req-3e9955eb-f08e-415c-bac9-67cf0c337521 service glance] Could not find project: 2387a046193b4ea98ad71ff3a32eb7e8.: keystone.exception.ProjectNotFound: Could not find project: 2387a046193b4ea98ad71ff3a32eb7e8. Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 21 23:09:15.078758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 23:09:15.080020 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 23:09:15.081375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application with Lock( Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._enter() Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.creator() Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application created_value = creator( Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 23:09:15.082529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 23:09:15.083914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 2387a046193b4ea98ad71ff3a32eb7e8. Jul 21 23:09:15.083914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: ERROR keystone.server.flask.application Jul 21 23:09:15.083914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1340/2662] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:14 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 112 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 21 23:09:15.110021 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bfcb2-48ed-47d5-8aa9-575107225e6f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:15.113989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03044fa1-86ff-4471-9aae-61c57c5407c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:15.114957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03044fa1-86ff-4471-9aae-61c57c5407c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:15.115311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03044fa1-86ff-4471-9aae-61c57c5407c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:15.116198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03044fa1-86ff-4471-9aae-61c57c5407c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:15.130419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03044fa1-86ff-4471-9aae-61c57c5407c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:15.131955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03044fa1-86ff-4471-9aae-61c57c5407c4 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:15.143661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1323/2663] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:15.574279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9955eb-f08e-415c-bac9-67cf0c337521 service glance] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:15.638671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9955eb-f08e-415c-bac9-67cf0c337521 service glance] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:15.643240 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:15.644107 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:15.644392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:15.644701 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] PATH_INFO: `/v3/users/f6d8aab1367a436a9a6f085f107f59ba` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:15.645902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d8aab1367a436a9a6f085f107f59ba)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:15.647806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:15.693922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d8aab1367a436a9a6f085f107f59ba'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:15.701880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d8aab1367a436a9a6f085f107f59ba'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:15.702481 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d8aab1367a436a9a6f085f107f59ba'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:15.709530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d8aab1367a436a9a6f085f107f59ba'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:15.710169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d8aab1367a436a9a6f085f107f59ba'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:15.727864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1341/2664] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:09:15 2022] DELETE /identity/v3/users/f6d8aab1367a436a9a6f085f107f59ba => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:09:15.732907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928deae-6ffc-46aa-941d-71054f64a4e2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:15.733204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928deae-6ffc-46aa-941d-71054f64a4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:15.733485 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928deae-6ffc-46aa-941d-71054f64a4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:15.751445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c928deae-6ffc-46aa-941d-71054f64a4e2 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:15.752681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c928deae-6ffc-46aa-941d-71054f64a4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:15.813892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1324/2665] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:15.824654 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:15.826243 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:15.836369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:15.889718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51194aa4-9484-4e45-8b4d-d3d802d70d9e admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:15.893771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9f047-8f5b-4875-9d04-c727d5fa8ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:15.894596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9f047-8f5b-4875-9d04-c727d5fa8ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:15.894919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9f047-8f5b-4875-9d04-c727d5fa8ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:15.895197 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9f047-8f5b-4875-9d04-c727d5fa8ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:15.907508 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9f047-8f5b-4875-9d04-c727d5fa8ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:15.909014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9f047-8f5b-4875-9d04-c727d5fa8ba4 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:15.918708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1342/2666] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:09:16.180316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928deae-6ffc-46aa-941d-71054f64a4e2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:16.181820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c928deae-6ffc-46aa-941d-71054f64a4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:16.196462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928deae-6ffc-46aa-941d-71054f64a4e2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:16.202066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:16.203135 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:16.203414 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:16.203690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] PATH_INFO: `/v3/projects/f80278af085f42c5b2254e60312444f0` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:16.205106 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f80278af085f42c5b2254e60312444f0)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:16.207005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:16.254468 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f80278af085f42c5b2254e60312444f0', 'request_id': 'req-ac897510-19d7-417e-8add-718c8205d5cf'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.260165 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f80278af085f42c5b2254e60312444f0', 'request_id': 'req-ac897510-19d7-417e-8add-718c8205d5cf'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.267763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f80278af085f42c5b2254e60312444f0', 'request_id': 'req-ac897510-19d7-417e-8add-718c8205d5cf'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.279834 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] The token cache is being invalidate because project f80278af085f42c5b2254e60312444f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:09:16.280259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.280949 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.282500 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1325/2667] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:09:16 2022] DELETE /identity/v3/projects/f80278af085f42c5b2254e60312444f0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:16.286434 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9f047-8f5b-4875-9d04-c727d5fa8ba4 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:16.297025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9f047-8f5b-4875-9d04-c727d5fa8ba4 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:16.361350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da086cd-4514-4e2c-836b-8d11a66bac31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:16.362238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da086cd-4514-4e2c-836b-8d11a66bac31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:16.362530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da086cd-4514-4e2c-836b-8d11a66bac31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:16.362811 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da086cd-4514-4e2c-836b-8d11a66bac31 admin admin] PATH_INFO: `/v3/users/ac5bbd1cc0c74c7296470651204657e6` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:16.364024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da086cd-4514-4e2c-836b-8d11a66bac31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac5bbd1cc0c74c7296470651204657e6)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:16.365999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da086cd-4514-4e2c-836b-8d11a66bac31 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:16.436387 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4da086cd-4514-4e2c-836b-8d11a66bac31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac5bbd1cc0c74c7296470651204657e6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.447731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4da086cd-4514-4e2c-836b-8d11a66bac31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac5bbd1cc0c74c7296470651204657e6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.447731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4da086cd-4514-4e2c-836b-8d11a66bac31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac5bbd1cc0c74c7296470651204657e6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.455770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4da086cd-4514-4e2c-836b-8d11a66bac31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac5bbd1cc0c74c7296470651204657e6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.456540 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4da086cd-4514-4e2c-836b-8d11a66bac31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac5bbd1cc0c74c7296470651204657e6'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.479463 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1343/2668] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:09:16 2022] DELETE /identity/v3/users/ac5bbd1cc0c74c7296470651204657e6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:16.782520 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:16.784195 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:16.803921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac897510-19d7-417e-8add-718c8205d5cf admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:16.834760 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:16.873613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd8f9c-ca62-4978-ad8f-725f520db372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:16.874488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd8f9c-ca62-4978-ad8f-725f520db372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:16.874771 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd8f9c-ca62-4978-ad8f-725f520db372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:16.875048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd8f9c-ca62-4978-ad8f-725f520db372 admin admin] PATH_INFO: `/v3/projects/85922b61cf2843b99bcec7e1b3bc545b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:16.876238 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd8f9c-ca62-4978-ad8f-725f520db372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85922b61cf2843b99bcec7e1b3bc545b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:16.878233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd8f9c-ca62-4978-ad8f-725f520db372 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:16.928544 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e2dd8f9c-ca62-4978-ad8f-725f520db372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85922b61cf2843b99bcec7e1b3bc545b', 'request_id': 'req-e2dd8f9c-ca62-4978-ad8f-725f520db372'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.934999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e2dd8f9c-ca62-4978-ad8f-725f520db372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85922b61cf2843b99bcec7e1b3bc545b', 'request_id': 'req-e2dd8f9c-ca62-4978-ad8f-725f520db372'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.942503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e2dd8f9c-ca62-4978-ad8f-725f520db372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85922b61cf2843b99bcec7e1b3bc545b', 'request_id': 'req-e2dd8f9c-ca62-4978-ad8f-725f520db372'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.955492 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e2dd8f9c-ca62-4978-ad8f-725f520db372 admin admin] The token cache is being invalidate because project 85922b61cf2843b99bcec7e1b3bc545b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:09:16.955778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e2dd8f9c-ca62-4978-ad8f-725f520db372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.956440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e2dd8f9c-ca62-4978-ad8f-725f520db372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:16.958017 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1326/2669] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:09:16 2022] DELETE /identity/v3/projects/85922b61cf2843b99bcec7e1b3bc545b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:16.962943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9951-ad91-442c-aee9-a4614a9a6e26 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:16.963933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9951-ad91-442c-aee9-a4614a9a6e26 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:16.964229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9951-ad91-442c-aee9-a4614a9a6e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:16.982559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-899c9951-ad91-442c-aee9-a4614a9a6e26 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:16.983770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-899c9951-ad91-442c-aee9-a4614a9a6e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:17.009302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:17.048129 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1344/2670] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:09:17.057752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a26d5d-77e4-4d16-bcce-d7ecd3906997 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.058040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a26d5d-77e4-4d16-bcce-d7ecd3906997 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.058319 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a26d5d-77e4-4d16-bcce-d7ecd3906997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.076589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-95a26d5d-77e4-4d16-bcce-d7ecd3906997 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:17.077809 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-95a26d5d-77e4-4d16-bcce-d7ecd3906997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:17.086348 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1327/2671] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:17.091572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9951-ad91-442c-aee9-a4614a9a6e26 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.102221 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9951-ad91-442c-aee9-a4614a9a6e26 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.107575 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024bd1e-c64a-468b-9faa-3a0bd22e31cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.108242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024bd1e-c64a-468b-9faa-3a0bd22e31cc admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.108561 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024bd1e-c64a-468b-9faa-3a0bd22e31cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.108842 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024bd1e-c64a-468b-9faa-3a0bd22e31cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.109583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024bd1e-c64a-468b-9faa-3a0bd22e31cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:09:17.109891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024bd1e-c64a-468b-9faa-3a0bd22e31cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.111475 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024bd1e-c64a-468b-9faa-3a0bd22e31cc admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.124985 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1345/2672] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:09:17.129184 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a26d5d-77e4-4d16-bcce-d7ecd3906997 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.138727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a26d5d-77e4-4d16-bcce-d7ecd3906997 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.145612 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eacebad-03c3-4fd3-9f6b-4bfb2387a4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.147153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacebad-03c3-4fd3-9f6b-4bfb2387a4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.147462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacebad-03c3-4fd3-9f6b-4bfb2387a4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.147718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacebad-03c3-4fd3-9f6b-4bfb2387a4e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.148434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eacebad-03c3-4fd3-9f6b-4bfb2387a4e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.150383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eacebad-03c3-4fd3-9f6b-4bfb2387a4e2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.178227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1328/2673] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:09:17 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:17.179975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024bd1e-c64a-468b-9faa-3a0bd22e31cc admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.181732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a024bd1e-c64a-468b-9faa-3a0bd22e31cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:17.182569 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eacebad-03c3-4fd3-9f6b-4bfb2387a4e2 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.245112 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eacebad-03c3-4fd3-9f6b-4bfb2387a4e2 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.250693 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4393073-9d82-4ebd-b8cc-f8592d6348b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.251580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4393073-9d82-4ebd-b8cc-f8592d6348b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.251868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4393073-9d82-4ebd-b8cc-f8592d6348b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.252148 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4393073-9d82-4ebd-b8cc-f8592d6348b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.252885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4393073-9d82-4ebd-b8cc-f8592d6348b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.254855 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4393073-9d82-4ebd-b8cc-f8592d6348b7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.271624 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024bd1e-c64a-468b-9faa-3a0bd22e31cc admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.277078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1329/2674] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:09:17 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:09:17.281149 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4393073-9d82-4ebd-b8cc-f8592d6348b7 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.290776 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4393073-9d82-4ebd-b8cc-f8592d6348b7 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.296318 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cfaad-59e8-4704-abc3-93f358c95605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.297048 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cfaad-59e8-4704-abc3-93f358c95605 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.297361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cfaad-59e8-4704-abc3-93f358c95605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.297586 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cfaad-59e8-4704-abc3-93f358c95605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.298399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45cfaad-59e8-4704-abc3-93f358c95605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.299927 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45cfaad-59e8-4704-abc3-93f358c95605 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.312860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1330/2675] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:17.313848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cfaad-59e8-4704-abc3-93f358c95605 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.323488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cfaad-59e8-4704-abc3-93f358c95605 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.328063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c977ab6-b064-4412-a72b-a0a64cb93459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.328728 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c977ab6-b064-4412-a72b-a0a64cb93459 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.329013 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c977ab6-b064-4412-a72b-a0a64cb93459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.329279 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c977ab6-b064-4412-a72b-a0a64cb93459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.329993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c977ab6-b064-4412-a72b-a0a64cb93459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.331598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c977ab6-b064-4412-a72b-a0a64cb93459 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.332998 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:17.336024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefc1e5-edbb-4802-ac9c-b1ab0ea41c46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.336720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefc1e5-edbb-4802-ac9c-b1ab0ea41c46 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.337011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefc1e5-edbb-4802-ac9c-b1ab0ea41c46 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.337296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefc1e5-edbb-4802-ac9c-b1ab0ea41c46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.339817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0cefc1e5-edbb-4802-ac9c-b1ab0ea41c46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:17.347961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1331/2676] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:09:17.349005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c977ab6-b064-4412-a72b-a0a64cb93459 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.352457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefc1e5-edbb-4802-ac9c-b1ab0ea41c46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.354312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefc1e5-edbb-4802-ac9c-b1ab0ea41c46 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.358763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c977ab6-b064-4412-a72b-a0a64cb93459 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.363479 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc76ad-fe09-424a-ba90-fb8c1ac4ef79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.364345 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc76ad-fe09-424a-ba90-fb8c1ac4ef79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.364659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc76ad-fe09-424a-ba90-fb8c1ac4ef79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.364938 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc76ad-fe09-424a-ba90-fb8c1ac4ef79 admin admin] PATH_INFO: `/v3/projects/e3de6fad6fb346bab8352b375d75f64f/users/73f51074294a47a7aa3529478a2b9bed/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.365350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1346/2677] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:17.380896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc76ad-fe09-424a-ba90-fb8c1ac4ef79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3de6fad6fb346bab8352b375d75f64f, user_id=73f51074294a47a7aa3529478a2b9bed, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.383583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc76ad-fe09-424a-ba90-fb8c1ac4ef79 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.401736 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1332/2678] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:17 2022] PUT /identity/v3/projects/e3de6fad6fb346bab8352b375d75f64f/users/73f51074294a47a7aa3529478a2b9bed/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:17.411714 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e9b84-b2fc-4ea6-b0a0-a7aeaf9aed5e None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.412010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e9b84-b2fc-4ea6-b0a0-a7aeaf9aed5e None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.412297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e9b84-b2fc-4ea6-b0a0-a7aeaf9aed5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.439123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-3e3e9b84-b2fc-4ea6-b0a0-a7aeaf9aed5e None None] MFA Rules not processed for user `73f51074294a47a7aa3529478a2b9bed`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:17.440333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-3e3e9b84-b2fc-4ea6-b0a0-a7aeaf9aed5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:17.456990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:17.505394 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1347/2679] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:09:17.510473 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc76ad-fe09-424a-ba90-fb8c1ac4ef79 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.572421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc76ad-fe09-424a-ba90-fb8c1ac4ef79 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.577499 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000403d6-4220-44df-8f7e-de08916377f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.578156 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000403d6-4220-44df-8f7e-de08916377f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.578438 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000403d6-4220-44df-8f7e-de08916377f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.578731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000403d6-4220-44df-8f7e-de08916377f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.579399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000403d6-4220-44df-8f7e-de08916377f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.581331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000403d6-4220-44df-8f7e-de08916377f2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.610664 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1333/2680] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:09:17 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:09:17.614971 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e9b84-b2fc-4ea6-b0a0-a7aeaf9aed5e None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.626081 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e9b84-b2fc-4ea6-b0a0-a7aeaf9aed5e None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.683168 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a037e33-7bcc-4aad-9648-c9a5c1e766a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.683841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a037e33-7bcc-4aad-9648-c9a5c1e766a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.684125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a037e33-7bcc-4aad-9648-c9a5c1e766a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.684396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a037e33-7bcc-4aad-9648-c9a5c1e766a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.685115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a037e33-7bcc-4aad-9648-c9a5c1e766a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.687001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a037e33-7bcc-4aad-9648-c9a5c1e766a7 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.709374 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1348/2681] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:09:17 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:17.714179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000403d6-4220-44df-8f7e-de08916377f2 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.725626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000403d6-4220-44df-8f7e-de08916377f2 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.730479 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869ceb7-d9e4-434e-b332-2b280bf9709b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.731149 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869ceb7-d9e4-434e-b332-2b280bf9709b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.731437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869ceb7-d9e4-434e-b332-2b280bf9709b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.731717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869ceb7-d9e4-434e-b332-2b280bf9709b admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.732444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869ceb7-d9e4-434e-b332-2b280bf9709b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.734093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869ceb7-d9e4-434e-b332-2b280bf9709b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.746773 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1334/2682] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:17.751363 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a037e33-7bcc-4aad-9648-c9a5c1e766a7 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.762791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a037e33-7bcc-4aad-9648-c9a5c1e766a7 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.767474 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fc382-2b8c-4ecf-8614-dd111520cc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.768385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fc382-2b8c-4ecf-8614-dd111520cc33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.768704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fc382-2b8c-4ecf-8614-dd111520cc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.768989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fc382-2b8c-4ecf-8614-dd111520cc33 admin admin] PATH_INFO: `/v3/projects/c560a7965a4542e8a74b08b4551ae5ed/users/dcd9fa4fc4d24d79aae178ae67bbb5ba/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.785097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fc382-2b8c-4ecf-8614-dd111520cc33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c560a7965a4542e8a74b08b4551ae5ed, user_id=dcd9fa4fc4d24d79aae178ae67bbb5ba, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.787741 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fc382-2b8c-4ecf-8614-dd111520cc33 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.802097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1349/2683] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:17 2022] PUT /identity/v3/projects/c560a7965a4542e8a74b08b4551ae5ed/users/dcd9fa4fc4d24d79aae178ae67bbb5ba/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:17.806177 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869ceb7-d9e4-434e-b332-2b280bf9709b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.816550 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869ceb7-d9e4-434e-b332-2b280bf9709b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.874604 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cf140-a2df-4838-a013-c7a245be7a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.875278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf140-a2df-4838-a013-c7a245be7a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.875561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf140-a2df-4838-a013-c7a245be7a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.875841 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf140-a2df-4838-a013-c7a245be7a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.876599 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cf140-a2df-4838-a013-c7a245be7a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.878206 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cf140-a2df-4838-a013-c7a245be7a50 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.889146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1335/2684] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:09:17.893452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fc382-2b8c-4ecf-8614-dd111520cc33 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:17.903639 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fc382-2b8c-4ecf-8614-dd111520cc33 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:17.908201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dd30e-d434-4ad6-8a5e-a01d5d388ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:17.909131 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dd30e-d434-4ad6-8a5e-a01d5d388ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.909420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dd30e-d434-4ad6-8a5e-a01d5d388ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.909698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dd30e-d434-4ad6-8a5e-a01d5d388ca6 admin admin] PATH_INFO: `/v3/projects/c560a7965a4542e8a74b08b4551ae5ed/users/dcd9fa4fc4d24d79aae178ae67bbb5ba/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.911671 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dd30e-d434-4ad6-8a5e-a01d5d388ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c560a7965a4542e8a74b08b4551ae5ed, user_id=dcd9fa4fc4d24d79aae178ae67bbb5ba, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:17.914364 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dd30e-d434-4ad6-8a5e-a01d5d388ca6 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:17.924504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1350/2685] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:17 2022] PUT /identity/v3/projects/c560a7965a4542e8a74b08b4551ae5ed/users/dcd9fa4fc4d24d79aae178ae67bbb5ba/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:17.935127 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db39d1-9ca5-4190-adc8-dd9ec9fae76c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:17.935426 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db39d1-9ca5-4190-adc8-dd9ec9fae76c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:17.935701 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db39d1-9ca5-4190-adc8-dd9ec9fae76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:17.961609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-33db39d1-9ca5-4190-adc8-dd9ec9fae76c None None] MFA Rules not processed for user `dcd9fa4fc4d24d79aae178ae67bbb5ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:17.962820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-33db39d1-9ca5-4190-adc8-dd9ec9fae76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:17.991573 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:18.031207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1336/2686] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:18.042607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dd30e-d434-4ad6-8a5e-a01d5d388ca6 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:18.044296 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-832dd30e-d434-4ad6-8a5e-a01d5d388ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:18.100615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dd30e-d434-4ad6-8a5e-a01d5d388ca6 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:18.104192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5b73c8-4563-4f53-bb6b-21682a27207c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:18.104863 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b73c8-4563-4f53-bb6b-21682a27207c service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:18.105144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b73c8-4563-4f53-bb6b-21682a27207c service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:18.105419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b73c8-4563-4f53-bb6b-21682a27207c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:18.114310 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5b73c8-4563-4f53-bb6b-21682a27207c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:18.115725 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5b73c8-4563-4f53-bb6b-21682a27207c service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:18.126104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1351/2687] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:18.189304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db39d1-9ca5-4190-adc8-dd9ec9fae76c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:18.252431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db39d1-9ca5-4190-adc8-dd9ec9fae76c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:18.257023 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:18.257889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:18.258174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:18.258444 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] PATH_INFO: `/v3/users/73f51074294a47a7aa3529478a2b9bed` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:18.259642 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=73f51074294a47a7aa3529478a2b9bed)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:18.262612 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:18.310748 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73f51074294a47a7aa3529478a2b9bed'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:18.317681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73f51074294a47a7aa3529478a2b9bed'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:18.330132 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73f51074294a47a7aa3529478a2b9bed'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:18.338723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73f51074294a47a7aa3529478a2b9bed'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:18.339659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73f51074294a47a7aa3529478a2b9bed'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:18.346139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1337/2688] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:09:18 2022] DELETE /identity/v3/users/73f51074294a47a7aa3529478a2b9bed => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:18.350978 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2ec0e-87e0-4b2f-9c18-889e3ac651d9 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:18.351509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2ec0e-87e0-4b2f-9c18-889e3ac651d9 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:18.351509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2ec0e-87e0-4b2f-9c18-889e3ac651d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:18.368902 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-d2d2ec0e-87e0-4b2f-9c18-889e3ac651d9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:18.371895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d2d2ec0e-87e0-4b2f-9c18-889e3ac651d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:18.381241 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:18.433935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1352/2689] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:18.443593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:18.445039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:18.497353 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:18.503678 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8925c70-b7d9-497c-80c5-4ac26a147ffc admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:18.508091 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eebc0c-0854-4dfe-b4b8-62847885f544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:18.508762 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eebc0c-0854-4dfe-b4b8-62847885f544 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:18.509051 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eebc0c-0854-4dfe-b4b8-62847885f544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:18.509329 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eebc0c-0854-4dfe-b4b8-62847885f544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:18.518889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eebc0c-0854-4dfe-b4b8-62847885f544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:18.520322 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eebc0c-0854-4dfe-b4b8-62847885f544 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:18.530563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1338/2690] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:18.806574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2ec0e-87e0-4b2f-9c18-889e3ac651d9 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:18.808791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d2d2ec0e-87e0-4b2f-9c18-889e3ac651d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:18.821460 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2ec0e-87e0-4b2f-9c18-889e3ac651d9 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:18.828051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:18.830788 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:18.831335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:18.831848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] PATH_INFO: `/v3/projects/e3de6fad6fb346bab8352b375d75f64f` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:18.833369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3de6fad6fb346bab8352b375d75f64f)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:18.838797 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:18.892082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3de6fad6fb346bab8352b375d75f64f', 'request_id': 'req-bf14aca4-58b3-440a-bbf1-47855925aed5'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:18.900144 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3de6fad6fb346bab8352b375d75f64f', 'request_id': 'req-bf14aca4-58b3-440a-bbf1-47855925aed5'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:18.906225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3de6fad6fb346bab8352b375d75f64f', 'request_id': 'req-bf14aca4-58b3-440a-bbf1-47855925aed5'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:18.917950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] The token cache is being invalidate because project e3de6fad6fb346bab8352b375d75f64f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:09:18.918246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:18.918908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:18.920469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1353/2691] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:09:18 2022] DELETE /identity/v3/projects/e3de6fad6fb346bab8352b375d75f64f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:18.924446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eebc0c-0854-4dfe-b4b8-62847885f544 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:18.935360 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eebc0c-0854-4dfe-b4b8-62847885f544 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:18.995203 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2afc0-65b1-40f3-a681-2d4c21f7ec7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:18.996063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2afc0-65b1-40f3-a681-2d4c21f7ec7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:18.996350 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2afc0-65b1-40f3-a681-2d4c21f7ec7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:18.996665 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2afc0-65b1-40f3-a681-2d4c21f7ec7c admin admin] PATH_INFO: `/v3/users/dcd9fa4fc4d24d79aae178ae67bbb5ba` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:18.997959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2afc0-65b1-40f3-a681-2d4c21f7ec7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcd9fa4fc4d24d79aae178ae67bbb5ba)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:19.000071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2afc0-65b1-40f3-a681-2d4c21f7ec7c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:19.047732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b0e2afc0-65b1-40f3-a681-2d4c21f7ec7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcd9fa4fc4d24d79aae178ae67bbb5ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:19.054989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b0e2afc0-65b1-40f3-a681-2d4c21f7ec7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcd9fa4fc4d24d79aae178ae67bbb5ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:19.066635 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b0e2afc0-65b1-40f3-a681-2d4c21f7ec7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcd9fa4fc4d24d79aae178ae67bbb5ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:19.073980 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b0e2afc0-65b1-40f3-a681-2d4c21f7ec7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcd9fa4fc4d24d79aae178ae67bbb5ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:19.074596 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b0e2afc0-65b1-40f3-a681-2d4c21f7ec7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcd9fa4fc4d24d79aae178ae67bbb5ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:19.080380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1339/2692] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:09:18 2022] DELETE /identity/v3/users/dcd9fa4fc4d24d79aae178ae67bbb5ba => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:09:19.349032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:19.350496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:19.364532 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14aca4-58b3-440a-bbf1-47855925aed5 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:19.423113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166203f9-872b-499a-8faa-828fb1683624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:19.423931 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166203f9-872b-499a-8faa-828fb1683624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:19.424220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166203f9-872b-499a-8faa-828fb1683624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:19.424530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166203f9-872b-499a-8faa-828fb1683624 admin admin] PATH_INFO: `/v3/projects/c560a7965a4542e8a74b08b4551ae5ed` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:19.425728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166203f9-872b-499a-8faa-828fb1683624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c560a7965a4542e8a74b08b4551ae5ed)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:19.427695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166203f9-872b-499a-8faa-828fb1683624 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:19.467838 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:19.475688 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-166203f9-872b-499a-8faa-828fb1683624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c560a7965a4542e8a74b08b4551ae5ed', 'request_id': 'req-166203f9-872b-499a-8faa-828fb1683624'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:19.483166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-166203f9-872b-499a-8faa-828fb1683624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c560a7965a4542e8a74b08b4551ae5ed', 'request_id': 'req-166203f9-872b-499a-8faa-828fb1683624'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:19.488903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-166203f9-872b-499a-8faa-828fb1683624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c560a7965a4542e8a74b08b4551ae5ed', 'request_id': 'req-166203f9-872b-499a-8faa-828fb1683624'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:19.500409 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-166203f9-872b-499a-8faa-828fb1683624 admin admin] The token cache is being invalidate because project c560a7965a4542e8a74b08b4551ae5ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:09:19.500727 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-166203f9-872b-499a-8faa-828fb1683624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:19.501345 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-166203f9-872b-499a-8faa-828fb1683624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:19.502866 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1354/2693] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:09:19 2022] DELETE /identity/v3/projects/c560a7965a4542e8a74b08b4551ae5ed => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:19.507752 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0458453-2f6e-4d76-935c-f9bbab82996c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:19.508228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0458453-2f6e-4d76-935c-f9bbab82996c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:19.508344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0458453-2f6e-4d76-935c-f9bbab82996c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:19.526846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f0458453-2f6e-4d76-935c-f9bbab82996c None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:19.528060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f0458453-2f6e-4d76-935c-f9bbab82996c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:19.592233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1340/2694] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:09:19.653959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb3004-b3a6-4855-9745-cbf66663ae30 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:19.654253 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb3004-b3a6-4855-9745-cbf66663ae30 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:19.654534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb3004-b3a6-4855-9745-cbf66663ae30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:19.673027 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-62fb3004-b3a6-4855-9745-cbf66663ae30 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:19.674329 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-62fb3004-b3a6-4855-9745-cbf66663ae30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:19.683086 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1355/2695] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:19.687377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0458453-2f6e-4d76-935c-f9bbab82996c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:19.697284 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0458453-2f6e-4d76-935c-f9bbab82996c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:19.702546 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b09171-4b1c-483d-a2fe-b965f04df86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:19.703203 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b09171-4b1c-483d-a2fe-b965f04df86d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:19.703490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b09171-4b1c-483d-a2fe-b965f04df86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:19.703771 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b09171-4b1c-483d-a2fe-b965f04df86d admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:19.704564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b09171-4b1c-483d-a2fe-b965f04df86d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:09:19.704895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b09171-4b1c-483d-a2fe-b965f04df86d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:19.706504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b09171-4b1c-483d-a2fe-b965f04df86d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:19.708901 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:19.719253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1341/2696] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 23:09:19.723324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb3004-b3a6-4855-9745-cbf66663ae30 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:19.733454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb3004-b3a6-4855-9745-cbf66663ae30 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:19.738145 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c2ad5-c2f5-42df-8fba-cfc8004d5de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:19.738829 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c2ad5-c2f5-42df-8fba-cfc8004d5de3 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:19.739110 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c2ad5-c2f5-42df-8fba-cfc8004d5de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:19.739386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c2ad5-c2f5-42df-8fba-cfc8004d5de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:19.740089 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c2ad5-c2f5-42df-8fba-cfc8004d5de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:19.742027 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c2ad5-c2f5-42df-8fba-cfc8004d5de3 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:19.768408 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1356/2697] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:09:19 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:19.772497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b09171-4b1c-483d-a2fe-b965f04df86d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:19.782431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b09171-4b1c-483d-a2fe-b965f04df86d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:19.844683 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40660a1c-37db-44a6-a225-64534d63d118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:19.845507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40660a1c-37db-44a6-a225-64534d63d118 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:19.845802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40660a1c-37db-44a6-a225-64534d63d118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:19.846082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40660a1c-37db-44a6-a225-64534d63d118 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:19.846876 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40660a1c-37db-44a6-a225-64534d63d118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:19.849041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40660a1c-37db-44a6-a225-64534d63d118 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:19.872237 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1342/2698] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:09:19 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:19.876571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c2ad5-c2f5-42df-8fba-cfc8004d5de3 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:19.888529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c2ad5-c2f5-42df-8fba-cfc8004d5de3 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:19.894057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ffe4f-f6dc-4809-8a91-871c05fd9f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:19.894751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ffe4f-f6dc-4809-8a91-871c05fd9f76 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:19.895042 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ffe4f-f6dc-4809-8a91-871c05fd9f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:19.895327 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ffe4f-f6dc-4809-8a91-871c05fd9f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:19.896718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ffe4f-f6dc-4809-8a91-871c05fd9f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:19.898335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ffe4f-f6dc-4809-8a91-871c05fd9f76 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:19.909647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1357/2699] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:19.913936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40660a1c-37db-44a6-a225-64534d63d118 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:19.924766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40660a1c-37db-44a6-a225-64534d63d118 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:19.929961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480daa5-5a19-4836-b232-8fe4308b3c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:19.930654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480daa5-5a19-4836-b232-8fe4308b3c11 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:19.930942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480daa5-5a19-4836-b232-8fe4308b3c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:19.931222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480daa5-5a19-4836-b232-8fe4308b3c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:19.932610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480daa5-5a19-4836-b232-8fe4308b3c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:19.934374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480daa5-5a19-4836-b232-8fe4308b3c11 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:19.946461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1343/2700] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:19.950982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ffe4f-f6dc-4809-8a91-871c05fd9f76 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:19.961520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ffe4f-f6dc-4809-8a91-871c05fd9f76 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:19.967670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae18a2c-85e6-44ee-b340-0d1667c90f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:19.968756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae18a2c-85e6-44ee-b340-0d1667c90f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:19.969095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae18a2c-85e6-44ee-b340-0d1667c90f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:19.969372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae18a2c-85e6-44ee-b340-0d1667c90f70 admin admin] PATH_INFO: `/v3/projects/0cdf99719e814f59a02ea5fd1326950e/users/968b1a734f474808b27fbb426faf8ca3/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:19.992791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae18a2c-85e6-44ee-b340-0d1667c90f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cdf99719e814f59a02ea5fd1326950e, user_id=968b1a734f474808b27fbb426faf8ca3, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:19.996231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae18a2c-85e6-44ee-b340-0d1667c90f70 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:20.023414 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1358/2701] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:19 2022] PUT /identity/v3/projects/0cdf99719e814f59a02ea5fd1326950e/users/968b1a734f474808b27fbb426faf8ca3/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:20.024926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33e486-172c-428c-b324-317673fe1db7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:20.025283 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33e486-172c-428c-b324-317673fe1db7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:20.025562 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33e486-172c-428c-b324-317673fe1db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:20.058698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4f33e486-172c-428c-b324-317673fe1db7 None None] MFA Rules not processed for user `968b1a734f474808b27fbb426faf8ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:20.060037 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4f33e486-172c-428c-b324-317673fe1db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:20.138519 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1344/2702] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:09:20.140249 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae18a2c-85e6-44ee-b340-0d1667c90f70 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:20.206596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae18a2c-85e6-44ee-b340-0d1667c90f70 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:20.211146 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5380921-3727-4906-8e02-89bd474af9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:20.211811 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5380921-3727-4906-8e02-89bd474af9da admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:20.212095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5380921-3727-4906-8e02-89bd474af9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:20.212373 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5380921-3727-4906-8e02-89bd474af9da admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:20.213611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5380921-3727-4906-8e02-89bd474af9da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:20.215533 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5380921-3727-4906-8e02-89bd474af9da admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:20.247324 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1359/2703] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:09:20 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:09:20.251548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33e486-172c-428c-b324-317673fe1db7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:20.266382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33e486-172c-428c-b324-317673fe1db7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:20.356074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245a318-245d-4240-a0e3-69e6bf1ef076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:20.357497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245a318-245d-4240-a0e3-69e6bf1ef076 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:20.357936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245a318-245d-4240-a0e3-69e6bf1ef076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:20.358255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245a318-245d-4240-a0e3-69e6bf1ef076 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:20.359311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245a318-245d-4240-a0e3-69e6bf1ef076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:20.361606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245a318-245d-4240-a0e3-69e6bf1ef076 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:20.372681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:20.394211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1345/2704] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:09:20 2022] POST /identity/v3/users => generated 300 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:20.398265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5380921-3727-4906-8e02-89bd474af9da admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:20.411282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5380921-3727-4906-8e02-89bd474af9da admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:20.420630 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac907a-944f-4ccd-b79f-c45d24139eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:20.421946 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac907a-944f-4ccd-b79f-c45d24139eda admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:20.423082 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac907a-944f-4ccd-b79f-c45d24139eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:20.423224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac907a-944f-4ccd-b79f-c45d24139eda admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:20.424301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac907a-944f-4ccd-b79f-c45d24139eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:20.426104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac907a-944f-4ccd-b79f-c45d24139eda admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:20.444570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1360/2705] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:20.446864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245a318-245d-4240-a0e3-69e6bf1ef076 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:20.460448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245a318-245d-4240-a0e3-69e6bf1ef076 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:20.467720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fba77-6dbf-4f1e-b15b-3302e8ecaef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:20.468952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fba77-6dbf-4f1e-b15b-3302e8ecaef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:20.469401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fba77-6dbf-4f1e-b15b-3302e8ecaef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:20.469858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fba77-6dbf-4f1e-b15b-3302e8ecaef5 admin admin] PATH_INFO: `/v3/projects/c1e66e0bbb5e4c5a97373dd7d02b1aba/users/c2e30d5fd37d453e8631f40b70f48add/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:20.488346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fba77-6dbf-4f1e-b15b-3302e8ecaef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e66e0bbb5e4c5a97373dd7d02b1aba, user_id=c2e30d5fd37d453e8631f40b70f48add, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:20.491380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fba77-6dbf-4f1e-b15b-3302e8ecaef5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:20.505320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1346/2706] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:20 2022] PUT /identity/v3/projects/c1e66e0bbb5e4c5a97373dd7d02b1aba/users/c2e30d5fd37d453e8631f40b70f48add/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:20.509556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac907a-944f-4ccd-b79f-c45d24139eda admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:20.519658 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac907a-944f-4ccd-b79f-c45d24139eda admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:20.555487 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:20.581139 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c35e59-ba99-4179-968a-173cedb5165f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:20.581748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c35e59-ba99-4179-968a-173cedb5165f admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:20.582035 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c35e59-ba99-4179-968a-173cedb5165f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:20.582318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c35e59-ba99-4179-968a-173cedb5165f admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:20.583050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c35e59-ba99-4179-968a-173cedb5165f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:20.584682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c35e59-ba99-4179-968a-173cedb5165f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:20.594252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1361/2707] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:20.598433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fba77-6dbf-4f1e-b15b-3302e8ecaef5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:20.609509 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fba77-6dbf-4f1e-b15b-3302e8ecaef5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:20.614231 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adfacc-95f4-45ae-a65e-892d5aafa0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:20.615126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adfacc-95f4-45ae-a65e-892d5aafa0f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:20.615416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adfacc-95f4-45ae-a65e-892d5aafa0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:20.615720 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adfacc-95f4-45ae-a65e-892d5aafa0f1 admin admin] PATH_INFO: `/v3/projects/c1e66e0bbb5e4c5a97373dd7d02b1aba/users/c2e30d5fd37d453e8631f40b70f48add/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:20.617771 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adfacc-95f4-45ae-a65e-892d5aafa0f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e66e0bbb5e4c5a97373dd7d02b1aba, user_id=c2e30d5fd37d453e8631f40b70f48add, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:20.621397 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adfacc-95f4-45ae-a65e-892d5aafa0f1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:20.632455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1347/2708] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:20 2022] PUT /identity/v3/projects/c1e66e0bbb5e4c5a97373dd7d02b1aba/users/c2e30d5fd37d453e8631f40b70f48add/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:20.643205 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5ab6c-42ae-47e4-a7d4-de2fac75312f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:20.643504 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5ab6c-42ae-47e4-a7d4-de2fac75312f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:20.643779 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5ab6c-42ae-47e4-a7d4-de2fac75312f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:20.680893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7ec5ab6c-42ae-47e4-a7d4-de2fac75312f None None] MFA Rules not processed for user `c2e30d5fd37d453e8631f40b70f48add`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:20.682736 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7ec5ab6c-42ae-47e4-a7d4-de2fac75312f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:20.766070 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1362/2709] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:20.778591 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adfacc-95f4-45ae-a65e-892d5aafa0f1 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:20.780886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-44adfacc-95f4-45ae-a65e-892d5aafa0f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:20.844852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adfacc-95f4-45ae-a65e-892d5aafa0f1 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:20.848423 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76f574-47ca-499f-9705-b8cdb9276ad3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:20.849099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f574-47ca-499f-9705-b8cdb9276ad3 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:20.849383 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f574-47ca-499f-9705-b8cdb9276ad3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:20.849658 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f574-47ca-499f-9705-b8cdb9276ad3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:20.859651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76f574-47ca-499f-9705-b8cdb9276ad3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:20.861141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76f574-47ca-499f-9705-b8cdb9276ad3 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:20.872611 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1348/2710] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:21.472564 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5ab6c-42ae-47e4-a7d4-de2fac75312f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:21.542146 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5ab6c-42ae-47e4-a7d4-de2fac75312f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:21.546720 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:21.547627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:21.547914 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:21.548201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] PATH_INFO: `/v3/users/968b1a734f474808b27fbb426faf8ca3` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:21.549427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=968b1a734f474808b27fbb426faf8ca3)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:21.551347 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:21.600251 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '968b1a734f474808b27fbb426faf8ca3'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:21.607673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '968b1a734f474808b27fbb426faf8ca3'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:21.608322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '968b1a734f474808b27fbb426faf8ca3'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:21.613585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:21.615732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '968b1a734f474808b27fbb426faf8ca3'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:21.616657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '968b1a734f474808b27fbb426faf8ca3'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:21.638047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1363/2711] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:09:21 2022] DELETE /identity/v3/users/968b1a734f474808b27fbb426faf8ca3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:21.643040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5aefc-9da6-4a7b-b131-baac9d7d2131 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:21.643558 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5aefc-9da6-4a7b-b131-baac9d7d2131 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:21.643663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5aefc-9da6-4a7b-b131-baac9d7d2131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:21.661541 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-16a5aefc-9da6-4a7b-b131-baac9d7d2131 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:21.662742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-16a5aefc-9da6-4a7b-b131-baac9d7d2131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:21.720419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1349/2712] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:21.730705 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:21.732121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:21.789372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144915f-640c-4446-9242-154900fe6d27 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:21.793729 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e34a25-8774-4fa3-905b-e121ea38a988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:21.794376 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e34a25-8774-4fa3-905b-e121ea38a988 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:21.794662 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e34a25-8774-4fa3-905b-e121ea38a988 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:21.794936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e34a25-8774-4fa3-905b-e121ea38a988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:21.803809 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e34a25-8774-4fa3-905b-e121ea38a988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:21.805265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e34a25-8774-4fa3-905b-e121ea38a988 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:21.814829 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1364/2713] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:22.089741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5aefc-9da6-4a7b-b131-baac9d7d2131 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:22.091946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-16a5aefc-9da6-4a7b-b131-baac9d7d2131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:22.109867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5aefc-9da6-4a7b-b131-baac9d7d2131 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:22.125056 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:22.126801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:22.127074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:22.127345 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] PATH_INFO: `/v3/projects/0cdf99719e814f59a02ea5fd1326950e` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:22.128632 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cdf99719e814f59a02ea5fd1326950e)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:22.130578 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:22.195347 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cdf99719e814f59a02ea5fd1326950e', 'request_id': 'req-97e0f98b-012d-4146-8a7e-79649fde3c7a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.200448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:22.203108 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cdf99719e814f59a02ea5fd1326950e', 'request_id': 'req-97e0f98b-012d-4146-8a7e-79649fde3c7a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.214819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cdf99719e814f59a02ea5fd1326950e', 'request_id': 'req-97e0f98b-012d-4146-8a7e-79649fde3c7a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.230911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] The token cache is being invalidate because project 0cdf99719e814f59a02ea5fd1326950e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:09:22.231317 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.232008 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.233652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1350/2714] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:09:22 2022] DELETE /identity/v3/projects/0cdf99719e814f59a02ea5fd1326950e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:22.237807 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e34a25-8774-4fa3-905b-e121ea38a988 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:22.252740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e34a25-8774-4fa3-905b-e121ea38a988 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:22.332903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96324120-2b28-410a-88f7-a0935a5126ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:22.334524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96324120-2b28-410a-88f7-a0935a5126ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:22.334841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96324120-2b28-410a-88f7-a0935a5126ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:22.335133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96324120-2b28-410a-88f7-a0935a5126ca admin admin] PATH_INFO: `/v3/users/c2e30d5fd37d453e8631f40b70f48add` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:22.336391 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96324120-2b28-410a-88f7-a0935a5126ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e30d5fd37d453e8631f40b70f48add)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:22.338909 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96324120-2b28-410a-88f7-a0935a5126ca admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:22.390293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-96324120-2b28-410a-88f7-a0935a5126ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e30d5fd37d453e8631f40b70f48add'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.399683 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-96324120-2b28-410a-88f7-a0935a5126ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e30d5fd37d453e8631f40b70f48add'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.401245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-96324120-2b28-410a-88f7-a0935a5126ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e30d5fd37d453e8631f40b70f48add'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.411372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-96324120-2b28-410a-88f7-a0935a5126ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e30d5fd37d453e8631f40b70f48add'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.412084 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-96324120-2b28-410a-88f7-a0935a5126ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e30d5fd37d453e8631f40b70f48add'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.435357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:22.441749 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1365/2715] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:09:22 2022] DELETE /identity/v3/users/c2e30d5fd37d453e8631f40b70f48add => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:22.739883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:22.741377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:22.755782 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0f98b-012d-4146-8a7e-79649fde3c7a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:22.819027 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cbc33-f460-4bf1-a3ce-fd797057eb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:22.819898 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cbc33-f460-4bf1-a3ce-fd797057eb22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:22.820175 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cbc33-f460-4bf1-a3ce-fd797057eb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:22.820443 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cbc33-f460-4bf1-a3ce-fd797057eb22 admin admin] PATH_INFO: `/v3/projects/c1e66e0bbb5e4c5a97373dd7d02b1aba` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:22.821654 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11cbc33-f460-4bf1-a3ce-fd797057eb22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e66e0bbb5e4c5a97373dd7d02b1aba)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:22.824094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11cbc33-f460-4bf1-a3ce-fd797057eb22 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:22.872138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e11cbc33-f460-4bf1-a3ce-fd797057eb22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e66e0bbb5e4c5a97373dd7d02b1aba', 'request_id': 'req-e11cbc33-f460-4bf1-a3ce-fd797057eb22'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.877812 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e11cbc33-f460-4bf1-a3ce-fd797057eb22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e66e0bbb5e4c5a97373dd7d02b1aba', 'request_id': 'req-e11cbc33-f460-4bf1-a3ce-fd797057eb22'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.886942 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e11cbc33-f460-4bf1-a3ce-fd797057eb22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e66e0bbb5e4c5a97373dd7d02b1aba', 'request_id': 'req-e11cbc33-f460-4bf1-a3ce-fd797057eb22'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.898910 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e11cbc33-f460-4bf1-a3ce-fd797057eb22 admin admin] The token cache is being invalidate because project c1e66e0bbb5e4c5a97373dd7d02b1aba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:09:22.899166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e11cbc33-f460-4bf1-a3ce-fd797057eb22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.899835 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e11cbc33-f460-4bf1-a3ce-fd797057eb22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:22.902432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1351/2716] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:09:22 2022] DELETE /identity/v3/projects/c1e66e0bbb5e4c5a97373dd7d02b1aba => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:22.907512 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af1bc0-2908-41b4-b45d-dd1ae5b073f2 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:22.908000 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af1bc0-2908-41b4-b45d-dd1ae5b073f2 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:22.908141 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af1bc0-2908-41b4-b45d-dd1ae5b073f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:22.926312 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-21af1bc0-2908-41b4-b45d-dd1ae5b073f2 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:22.927534 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-21af1bc0-2908-41b4-b45d-dd1ae5b073f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:22.997011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1366/2717] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:22.998803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae503aea-461a-48e1-93f7-7d2e0c2f88a8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:22.999122 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae503aea-461a-48e1-93f7-7d2e0c2f88a8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:22.999402 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae503aea-461a-48e1-93f7-7d2e0c2f88a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.018125 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ae503aea-461a-48e1-93f7-7d2e0c2f88a8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:23.019335 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ae503aea-461a-48e1-93f7-7d2e0c2f88a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:23.028074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1352/2718] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:09:23.033083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af1bc0-2908-41b4-b45d-dd1ae5b073f2 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.043227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af1bc0-2908-41b4-b45d-dd1ae5b073f2 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.047781 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a029416-46c7-4515-b047-f8c344263b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.048437 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a029416-46c7-4515-b047-f8c344263b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.048760 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a029416-46c7-4515-b047-f8c344263b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.049024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a029416-46c7-4515-b047-f8c344263b64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.049757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a029416-46c7-4515-b047-f8c344263b64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:09:23.050064 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a029416-46c7-4515-b047-f8c344263b64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.051640 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a029416-46c7-4515-b047-f8c344263b64 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.063492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1367/2719] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:09:23.067973 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae503aea-461a-48e1-93f7-7d2e0c2f88a8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.077983 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae503aea-461a-48e1-93f7-7d2e0c2f88a8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.082703 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1544c-418b-40ed-bd55-440f94a55a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.083357 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1544c-418b-40ed-bd55-440f94a55a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.083643 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1544c-418b-40ed-bd55-440f94a55a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.083921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1544c-418b-40ed-bd55-440f94a55a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.084656 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1544c-418b-40ed-bd55-440f94a55a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.086564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1544c-418b-40ed-bd55-440f94a55a58 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.112200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1353/2720] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:09:23 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:23.116322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a029416-46c7-4515-b047-f8c344263b64 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.126804 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a029416-46c7-4515-b047-f8c344263b64 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.181378 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb9527-86c0-4c4d-8549-a6ccd9dcb043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.182047 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb9527-86c0-4c4d-8549-a6ccd9dcb043 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.182331 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb9527-86c0-4c4d-8549-a6ccd9dcb043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.182611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb9527-86c0-4c4d-8549-a6ccd9dcb043 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.183814 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb9527-86c0-4c4d-8549-a6ccd9dcb043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.185758 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb9527-86c0-4c4d-8549-a6ccd9dcb043 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.206105 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1368/2721] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:09:23 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:23.210331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1544c-418b-40ed-bd55-440f94a55a58 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.220594 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1544c-418b-40ed-bd55-440f94a55a58 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.225126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ddd87-6a87-4a70-af59-890df2bc726a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.225778 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ddd87-6a87-4a70-af59-890df2bc726a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.226061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ddd87-6a87-4a70-af59-890df2bc726a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.226339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ddd87-6a87-4a70-af59-890df2bc726a admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.227060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ddd87-6a87-4a70-af59-890df2bc726a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.228744 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ddd87-6a87-4a70-af59-890df2bc726a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.239914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1354/2722] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:23.243896 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb9527-86c0-4c4d-8549-a6ccd9dcb043 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.254997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb9527-86c0-4c4d-8549-a6ccd9dcb043 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.259775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46630d67-c13c-4e45-89bc-5f0304bef541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.260440 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46630d67-c13c-4e45-89bc-5f0304bef541 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.260772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46630d67-c13c-4e45-89bc-5f0304bef541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.261795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46630d67-c13c-4e45-89bc-5f0304bef541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.263125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46630d67-c13c-4e45-89bc-5f0304bef541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.265193 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46630d67-c13c-4e45-89bc-5f0304bef541 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.275283 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1369/2723] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:23.279299 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ddd87-6a87-4a70-af59-890df2bc726a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.289198 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ddd87-6a87-4a70-af59-890df2bc726a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.293847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cddb4d-3461-4152-8bad-d5db31d1dc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.294730 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cddb4d-3461-4152-8bad-d5db31d1dc35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.295014 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cddb4d-3461-4152-8bad-d5db31d1dc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.295295 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cddb4d-3461-4152-8bad-d5db31d1dc35 admin admin] PATH_INFO: `/v3/projects/31534144acad477db46d2f0fd7c659b7/users/46301cc49f6d4ce9ade9428939e16d89/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.310926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cddb4d-3461-4152-8bad-d5db31d1dc35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31534144acad477db46d2f0fd7c659b7, user_id=46301cc49f6d4ce9ade9428939e16d89, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.313695 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cddb4d-3461-4152-8bad-d5db31d1dc35 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.328445 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1355/2724] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:23 2022] PUT /identity/v3/projects/31534144acad477db46d2f0fd7c659b7/users/46301cc49f6d4ce9ade9428939e16d89/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:23.340014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16df5f-6fff-4d3a-b287-10a40e48ede7 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.340302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16df5f-6fff-4d3a-b287-10a40e48ede7 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.340610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16df5f-6fff-4d3a-b287-10a40e48ede7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.369289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-8d16df5f-6fff-4d3a-b287-10a40e48ede7 None None] MFA Rules not processed for user `46301cc49f6d4ce9ade9428939e16d89`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:23.371326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-8d16df5f-6fff-4d3a-b287-10a40e48ede7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:23.429964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:23.441077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1370/2725] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:23.445456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cddb4d-3461-4152-8bad-d5db31d1dc35 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.456063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:23.510310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cddb4d-3461-4152-8bad-d5db31d1dc35 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.516157 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484590f-2421-4f54-aba4-77969f2e0ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.516891 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484590f-2421-4f54-aba4-77969f2e0ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.517181 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484590f-2421-4f54-aba4-77969f2e0ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.517461 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484590f-2421-4f54-aba4-77969f2e0ff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.518153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484590f-2421-4f54-aba4-77969f2e0ff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.520072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484590f-2421-4f54-aba4-77969f2e0ff1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.546563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1356/2726] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:09:23 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:23.550848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16df5f-6fff-4d3a-b287-10a40e48ede7 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.562618 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16df5f-6fff-4d3a-b287-10a40e48ede7 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.624848 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67aca3-d13d-445a-b396-e9f1e3e30765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.625533 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67aca3-d13d-445a-b396-e9f1e3e30765 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.625819 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67aca3-d13d-445a-b396-e9f1e3e30765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.626101 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67aca3-d13d-445a-b396-e9f1e3e30765 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.626803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67aca3-d13d-445a-b396-e9f1e3e30765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.628901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67aca3-d13d-445a-b396-e9f1e3e30765 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.650494 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1371/2727] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:09:23 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:23.655159 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484590f-2421-4f54-aba4-77969f2e0ff1 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.666630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484590f-2421-4f54-aba4-77969f2e0ff1 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.673626 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01d351-a3a5-42c5-a353-9a3253172519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.674300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01d351-a3a5-42c5-a353-9a3253172519 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.674583 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01d351-a3a5-42c5-a353-9a3253172519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.674860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01d351-a3a5-42c5-a353-9a3253172519 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.675588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01d351-a3a5-42c5-a353-9a3253172519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.677257 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01d351-a3a5-42c5-a353-9a3253172519 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.690412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1357/2728] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:23.694212 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01d351-a3a5-42c5-a353-9a3253172519 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.694381 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67aca3-d13d-445a-b396-e9f1e3e30765 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.696039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5a67aca3-d13d-445a-b396-e9f1e3e30765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:23.704652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01d351-a3a5-42c5-a353-9a3253172519 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.709432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded3eaa-c622-4c44-8a6c-244877e32d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.710074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded3eaa-c622-4c44-8a6c-244877e32d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.710352 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded3eaa-c622-4c44-8a6c-244877e32d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.710624 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded3eaa-c622-4c44-8a6c-244877e32d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.711340 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded3eaa-c622-4c44-8a6c-244877e32d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.712961 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded3eaa-c622-4c44-8a6c-244877e32d0c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.723481 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1358/2729] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:23.727395 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded3eaa-c622-4c44-8a6c-244877e32d0c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.737577 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded3eaa-c622-4c44-8a6c-244877e32d0c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.742488 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fa8e4-f1e9-472e-8902-5474c8c079e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.743412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fa8e4-f1e9-472e-8902-5474c8c079e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.743700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fa8e4-f1e9-472e-8902-5474c8c079e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.743986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fa8e4-f1e9-472e-8902-5474c8c079e1 admin admin] PATH_INFO: `/v3/projects/7353a84e2e924cf9a4822cdbe89b8972/users/3c9e117fb02f4e1c8497a01b155eaf1b/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.759836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fa8e4-f1e9-472e-8902-5474c8c079e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7353a84e2e924cf9a4822cdbe89b8972, user_id=3c9e117fb02f4e1c8497a01b155eaf1b, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.760013 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67aca3-d13d-445a-b396-e9f1e3e30765 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.763055 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fa8e4-f1e9-472e-8902-5474c8c079e1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.763772 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3915d-4468-49fb-b84c-5656b4694c7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.764621 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3915d-4468-49fb-b84c-5656b4694c7a service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.764919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3915d-4468-49fb-b84c-5656b4694c7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.765200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3915d-4468-49fb-b84c-5656b4694c7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.767050 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-54a3915d-4468-49fb-b84c-5656b4694c7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:23.776605 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1359/2730] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:23 2022] PUT /identity/v3/projects/7353a84e2e924cf9a4822cdbe89b8972/users/3c9e117fb02f4e1c8497a01b155eaf1b/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:09:23.779692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3915d-4468-49fb-b84c-5656b4694c7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.781160 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3915d-4468-49fb-b84c-5656b4694c7a service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.788346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e4e2e-bf9c-4c66-bf49-6b6a00558d55 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.788675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e4e2e-bf9c-4c66-bf49-6b6a00558d55 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.788956 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e4e2e-bf9c-4c66-bf49-6b6a00558d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.791049 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1372/2731] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:23.815293 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8e3e4e2e-bf9c-4c66-bf49-6b6a00558d55 None None] MFA Rules not processed for user `3c9e117fb02f4e1c8497a01b155eaf1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:23.816591 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8e3e4e2e-bf9c-4c66-bf49-6b6a00558d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:23.888153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1360/2732] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:09:23.892645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3915d-4468-49fb-b84c-5656b4694c7a service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:23.902824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3915d-4468-49fb-b84c-5656b4694c7a service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:23.957792 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f736a95-33ea-4c33-967b-687119113821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:23.958591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f736a95-33ea-4c33-967b-687119113821 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:23.958943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f736a95-33ea-4c33-967b-687119113821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:23.959288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f736a95-33ea-4c33-967b-687119113821 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:23.960045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f736a95-33ea-4c33-967b-687119113821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:23.962029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f736a95-33ea-4c33-967b-687119113821 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:23.991885 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1373/2733] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:09:23 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:23.993504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e4e2e-bf9c-4c66-bf49-6b6a00558d55 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:24.005291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e4e2e-bf9c-4c66-bf49-6b6a00558d55 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:24.038491 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:24.064019 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962993b-3a8e-4714-9318-10faac706587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:24.064724 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962993b-3a8e-4714-9318-10faac706587 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:24.065070 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962993b-3a8e-4714-9318-10faac706587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:24.065348 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962993b-3a8e-4714-9318-10faac706587 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:24.066041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962993b-3a8e-4714-9318-10faac706587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:24.067970 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962993b-3a8e-4714-9318-10faac706587 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:24.087152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1361/2734] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:09:23 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:09:24.091656 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f736a95-33ea-4c33-967b-687119113821 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:24.102366 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f736a95-33ea-4c33-967b-687119113821 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:24.110305 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe81f7-d379-4ed6-aaf9-6d07e6d2a2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:24.111010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe81f7-d379-4ed6-aaf9-6d07e6d2a2db admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:24.111321 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe81f7-d379-4ed6-aaf9-6d07e6d2a2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:24.111610 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe81f7-d379-4ed6-aaf9-6d07e6d2a2db admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:24.112432 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe81f7-d379-4ed6-aaf9-6d07e6d2a2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:24.114153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe81f7-d379-4ed6-aaf9-6d07e6d2a2db admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:24.127533 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1374/2735] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:24.131826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962993b-3a8e-4714-9318-10faac706587 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:24.142689 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962993b-3a8e-4714-9318-10faac706587 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:24.147530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a174d-5255-49fc-9a0c-d2384b920c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:24.148460 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a174d-5255-49fc-9a0c-d2384b920c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:24.148779 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a174d-5255-49fc-9a0c-d2384b920c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:24.149058 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a174d-5255-49fc-9a0c-d2384b920c88 admin admin] PATH_INFO: `/v3/projects/b719384d2b9746f88bc816b8b90987c2/users/4696ac2517e44acaaee9f3a577c4b652/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:24.165216 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a174d-5255-49fc-9a0c-d2384b920c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b719384d2b9746f88bc816b8b90987c2, user_id=4696ac2517e44acaaee9f3a577c4b652, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:24.168432 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a174d-5255-49fc-9a0c-d2384b920c88 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:24.183725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1362/2736] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:24 2022] PUT /identity/v3/projects/b719384d2b9746f88bc816b8b90987c2/users/4696ac2517e44acaaee9f3a577c4b652/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:24.187450 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe81f7-d379-4ed6-aaf9-6d07e6d2a2db admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:24.192934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:24.199533 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe81f7-d379-4ed6-aaf9-6d07e6d2a2db admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:24.256947 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33782411-5428-4b74-96d2-b7f214288076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:24.257636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33782411-5428-4b74-96d2-b7f214288076 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:24.257922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33782411-5428-4b74-96d2-b7f214288076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:24.258201 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33782411-5428-4b74-96d2-b7f214288076 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:24.258950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33782411-5428-4b74-96d2-b7f214288076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:24.260609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33782411-5428-4b74-96d2-b7f214288076 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:24.272315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1375/2737] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:24.277309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a174d-5255-49fc-9a0c-d2384b920c88 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:24.288092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a174d-5255-49fc-9a0c-d2384b920c88 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:24.293188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695c1e3-0677-48f9-8d21-c0779838a0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:24.294151 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695c1e3-0677-48f9-8d21-c0779838a0d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:24.294434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695c1e3-0677-48f9-8d21-c0779838a0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:24.294859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695c1e3-0677-48f9-8d21-c0779838a0d3 admin admin] PATH_INFO: `/v3/projects/b719384d2b9746f88bc816b8b90987c2/users/4696ac2517e44acaaee9f3a577c4b652/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:24.296928 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695c1e3-0677-48f9-8d21-c0779838a0d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b719384d2b9746f88bc816b8b90987c2, user_id=4696ac2517e44acaaee9f3a577c4b652, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:24.299630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695c1e3-0677-48f9-8d21-c0779838a0d3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:24.311101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1363/2738] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:24 2022] PUT /identity/v3/projects/b719384d2b9746f88bc816b8b90987c2/users/4696ac2517e44acaaee9f3a577c4b652/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:09:24.321987 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b77a1-6667-4e84-a020-66c9c49f4102 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:24.322380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b77a1-6667-4e84-a020-66c9c49f4102 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:24.322734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b77a1-6667-4e84-a020-66c9c49f4102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:24.352119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-587b77a1-6667-4e84-a020-66c9c49f4102 None None] MFA Rules not processed for user `4696ac2517e44acaaee9f3a577c4b652`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:24.353379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-587b77a1-6667-4e84-a020-66c9c49f4102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:24.427928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1376/2739] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:24.439415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695c1e3-0677-48f9-8d21-c0779838a0d3 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:24.441207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a695c1e3-0677-48f9-8d21-c0779838a0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:24.509856 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695c1e3-0677-48f9-8d21-c0779838a0d3 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:24.516292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e6f09-80cb-4d72-8915-266a88a654ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:24.517327 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e6f09-80cb-4d72-8915-266a88a654ef service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:24.518798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e6f09-80cb-4d72-8915-266a88a654ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:24.519088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e6f09-80cb-4d72-8915-266a88a654ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:24.591765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e6f09-80cb-4d72-8915-266a88a654ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:24.593416 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e6f09-80cb-4d72-8915-266a88a654ef service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:24.603123 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1364/2740] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:26.164245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b77a1-6667-4e84-a020-66c9c49f4102 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:26.174282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b77a1-6667-4e84-a020-66c9c49f4102 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:26.178140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51cbe4-8b02-432a-917b-b66c57bb7d87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:26.178800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51cbe4-8b02-432a-917b-b66c57bb7d87 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:26.179087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51cbe4-8b02-432a-917b-b66c57bb7d87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:26.179367 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51cbe4-8b02-432a-917b-b66c57bb7d87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:26.237581 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51cbe4-8b02-432a-917b-b66c57bb7d87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:26.239023 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51cbe4-8b02-432a-917b-b66c57bb7d87 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:26.249170 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1377/2741] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:26.874121 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e6f09-80cb-4d72-8915-266a88a654ef service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:26.894313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e6f09-80cb-4d72-8915-266a88a654ef service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:26.907503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210de63-edf6-4c27-93d6-3d3b444ae126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:26.908322 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210de63-edf6-4c27-93d6-3d3b444ae126 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:26.909036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210de63-edf6-4c27-93d6-3d3b444ae126 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:26.909495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210de63-edf6-4c27-93d6-3d3b444ae126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:26.927437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210de63-edf6-4c27-93d6-3d3b444ae126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:26.929472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210de63-edf6-4c27-93d6-3d3b444ae126 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:26.947935 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51cbe4-8b02-432a-917b-b66c57bb7d87 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:26.950855 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4a51cbe4-8b02-432a-917b-b66c57bb7d87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:26.959610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1365/2742] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:26.990948 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:27.021325 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210de63-edf6-4c27-93d6-3d3b444ae126 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:27.022649 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9210de63-edf6-4c27-93d6-3d3b444ae126 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:27.044186 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51cbe4-8b02-432a-917b-b66c57bb7d87 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:27.060104 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435bd87-c582-4a82-ab35-4a5189dc7722 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:27.060949 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435bd87-c582-4a82-ab35-4a5189dc7722 service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:27.061192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435bd87-c582-4a82-ab35-4a5189dc7722 service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:27.061466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435bd87-c582-4a82-ab35-4a5189dc7722 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:27.065668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4435bd87-c582-4a82-ab35-4a5189dc7722 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:27.079535 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:27.080044 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435bd87-c582-4a82-ab35-4a5189dc7722 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:27.081906 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435bd87-c582-4a82-ab35-4a5189dc7722 service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:27.093859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1378/2743] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:09:27.131798 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210de63-edf6-4c27-93d6-3d3b444ae126 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:27.140993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669256e-35d8-4ceb-8d5c-6408a0579793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:27.142053 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669256e-35d8-4ceb-8d5c-6408a0579793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:27.142402 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669256e-35d8-4ceb-8d5c-6408a0579793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:27.142801 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669256e-35d8-4ceb-8d5c-6408a0579793 admin admin] PATH_INFO: `/v3/users/f93bd44f2fbc4be4be905c8b8ae0e343` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:27.144353 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669256e-35d8-4ceb-8d5c-6408a0579793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f93bd44f2fbc4be4be905c8b8ae0e343)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:27.146727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669256e-35d8-4ceb-8d5c-6408a0579793 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:27.196505 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5669256e-35d8-4ceb-8d5c-6408a0579793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f93bd44f2fbc4be4be905c8b8ae0e343'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:27.204965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5669256e-35d8-4ceb-8d5c-6408a0579793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f93bd44f2fbc4be4be905c8b8ae0e343'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:27.217922 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5669256e-35d8-4ceb-8d5c-6408a0579793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f93bd44f2fbc4be4be905c8b8ae0e343'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:27.225957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5669256e-35d8-4ceb-8d5c-6408a0579793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f93bd44f2fbc4be4be905c8b8ae0e343'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:27.226650 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5669256e-35d8-4ceb-8d5c-6408a0579793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f93bd44f2fbc4be4be905c8b8ae0e343'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:27.233702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1366/2744] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:09:27 2022] DELETE /identity/v3/users/f93bd44f2fbc4be4be905c8b8ae0e343 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:09:27.408282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435bd87-c582-4a82-ab35-4a5189dc7722 service placement] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:27.420255 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435bd87-c582-4a82-ab35-4a5189dc7722 service placement] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:27.483237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:27.484092 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:27.484379 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:27.484698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345 admin admin] PATH_INFO: `/v3/projects/0b2f768290ea43cb98b058067268674d` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:27.485886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2f768290ea43cb98b058067268674d)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:27.487781 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:27.532524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2f768290ea43cb98b058067268674d', 'request_id': 'req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:27.539591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2f768290ea43cb98b058067268674d', 'request_id': 'req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:27.545835 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2f768290ea43cb98b058067268674d', 'request_id': 'req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:27.556564 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345 admin admin] The token cache is being invalidate because project 0b2f768290ea43cb98b058067268674d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:09:27.556856 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:27.558224 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-99ba5f6e-8bc0-46f8-b999-f8e1daba3345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:27.559665 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1379/2745] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:09:27 2022] DELETE /identity/v3/projects/0b2f768290ea43cb98b058067268674d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:27.565703 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00e8e9-8372-40c0-a6b8-9199c6683eb8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:27.566024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00e8e9-8372-40c0-a6b8-9199c6683eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:27.566304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00e8e9-8372-40c0-a6b8-9199c6683eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:27.584010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4f00e8e9-8372-40c0-a6b8-9199c6683eb8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:27.585263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4f00e8e9-8372-40c0-a6b8-9199c6683eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:27.645511 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1367/2746] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:09:27.654528 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6424b16-1479-48b6-9e11-7d052c3354ed None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:27.654825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6424b16-1479-48b6-9e11-7d052c3354ed None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:27.655113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6424b16-1479-48b6-9e11-7d052c3354ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:27.679127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-b6424b16-1479-48b6-9e11-7d052c3354ed None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:27.680343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b6424b16-1479-48b6-9e11-7d052c3354ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:27.688330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1380/2747] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:27.692465 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00e8e9-8372-40c0-a6b8-9199c6683eb8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:27.702064 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00e8e9-8372-40c0-a6b8-9199c6683eb8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:27.706588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b792639-57d7-47a1-af91-7fb52939ae16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:27.707240 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b792639-57d7-47a1-af91-7fb52939ae16 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:27.707523 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b792639-57d7-47a1-af91-7fb52939ae16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:27.707797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b792639-57d7-47a1-af91-7fb52939ae16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:27.708657 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b792639-57d7-47a1-af91-7fb52939ae16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:09:27.708995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b792639-57d7-47a1-af91-7fb52939ae16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:27.710580 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b792639-57d7-47a1-af91-7fb52939ae16 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:27.723203 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1368/2748] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:09:27.726972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6424b16-1479-48b6-9e11-7d052c3354ed None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:27.737257 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6424b16-1479-48b6-9e11-7d052c3354ed None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:27.741802 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6ced5-00d5-43b1-8f0f-77d5f45d7f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:27.742465 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6ced5-00d5-43b1-8f0f-77d5f45d7f52 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:27.742782 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6ced5-00d5-43b1-8f0f-77d5f45d7f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:27.743064 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6ced5-00d5-43b1-8f0f-77d5f45d7f52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:27.743764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6ced5-00d5-43b1-8f0f-77d5f45d7f52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:27.746210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6ced5-00d5-43b1-8f0f-77d5f45d7f52 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:27.772748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1381/2749] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:09:27 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:27.777464 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b792639-57d7-47a1-af91-7fb52939ae16 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:27.787568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b792639-57d7-47a1-af91-7fb52939ae16 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:27.822153 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:27.851755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16775095-b0e7-4e3f-9d1d-4b50f992efb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:27.852466 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775095-b0e7-4e3f-9d1d-4b50f992efb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:27.852820 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775095-b0e7-4e3f-9d1d-4b50f992efb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:27.853099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16775095-b0e7-4e3f-9d1d-4b50f992efb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:27.853679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16775095-b0e7-4e3f-9d1d-4b50f992efb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:27.856285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16775095-b0e7-4e3f-9d1d-4b50f992efb0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:27.876093 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1369/2750] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:09:27 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:27.880145 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6ced5-00d5-43b1-8f0f-77d5f45d7f52 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:27.890385 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6ced5-00d5-43b1-8f0f-77d5f45d7f52 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:27.894864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe175a-2a52-44a9-b193-1755ec90fd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:27.895521 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe175a-2a52-44a9-b193-1755ec90fd19 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:27.895806 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe175a-2a52-44a9-b193-1755ec90fd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:27.896081 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe175a-2a52-44a9-b193-1755ec90fd19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:27.896831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe175a-2a52-44a9-b193-1755ec90fd19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:27.898410 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe175a-2a52-44a9-b193-1755ec90fd19 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:27.909634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1382/2751] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:27.913341 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16775095-b0e7-4e3f-9d1d-4b50f992efb0 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:27.925117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16775095-b0e7-4e3f-9d1d-4b50f992efb0 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:27.929971 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf823cbc-cc03-437f-880b-061713b66e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:27.930659 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf823cbc-cc03-437f-880b-061713b66e2b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:27.930951 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf823cbc-cc03-437f-880b-061713b66e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:27.931229 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf823cbc-cc03-437f-880b-061713b66e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:27.931978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf823cbc-cc03-437f-880b-061713b66e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:27.933696 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf823cbc-cc03-437f-880b-061713b66e2b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:27.944084 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1370/2752] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:27.948709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe175a-2a52-44a9-b193-1755ec90fd19 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:27.955146 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:27.960508 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe175a-2a52-44a9-b193-1755ec90fd19 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:27.967692 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6e442-e8f1-4c55-a69c-e870b69501f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:27.968821 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6e442-e8f1-4c55-a69c-e870b69501f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:27.969490 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6e442-e8f1-4c55-a69c-e870b69501f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:27.970064 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6e442-e8f1-4c55-a69c-e870b69501f4 admin admin] PATH_INFO: `/v3/projects/37eee429bcbb4cdabc59a2e8a23da997/users/8a7e4fed38914f91beae1c094d73163f/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:27.985858 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6e442-e8f1-4c55-a69c-e870b69501f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37eee429bcbb4cdabc59a2e8a23da997, user_id=8a7e4fed38914f91beae1c094d73163f, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:27.988413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6e442-e8f1-4c55-a69c-e870b69501f4 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:28.005599 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1383/2753] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:27 2022] PUT /identity/v3/projects/37eee429bcbb4cdabc59a2e8a23da997/users/8a7e4fed38914f91beae1c094d73163f/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:28.007630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574a381-16a5-4a69-a3d3-720266e9c237 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:28.008011 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574a381-16a5-4a69-a3d3-720266e9c237 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:28.008296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574a381-16a5-4a69-a3d3-720266e9c237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:28.027940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9574a381-16a5-4a69-a3d3-720266e9c237 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:28.029184 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9574a381-16a5-4a69-a3d3-720266e9c237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:28.087881 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1371/2754] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:28.098281 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6e442-e8f1-4c55-a69c-e870b69501f4 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:28.099958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bcc6e442-e8f1-4c55-a69c-e870b69501f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:28.161318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6e442-e8f1-4c55-a69c-e870b69501f4 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:28.165697 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaeb9a8-bc8d-4242-b69c-c7bf336c69a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:28.165985 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaeb9a8-bc8d-4242-b69c-c7bf336c69a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:28.166099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaeb9a8-bc8d-4242-b69c-c7bf336c69a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:28.166308 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaeb9a8-bc8d-4242-b69c-c7bf336c69a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:28.176113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaeb9a8-bc8d-4242-b69c-c7bf336c69a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:28.177679 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaeb9a8-bc8d-4242-b69c-c7bf336c69a7 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:28.188194 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1384/2755] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:31.034451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574a381-16a5-4a69-a3d3-720266e9c237 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:31.036052 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9574a381-16a5-4a69-a3d3-720266e9c237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:31.048939 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574a381-16a5-4a69-a3d3-720266e9c237 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:31.054194 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:31.055602 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:31.056098 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:31.056612 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] PATH_INFO: `/v3/users/46301cc49f6d4ce9ade9428939e16d89` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:31.058859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46301cc49f6d4ce9ade9428939e16d89)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:31.061504 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:31.121168 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46301cc49f6d4ce9ade9428939e16d89'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:31.129427 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46301cc49f6d4ce9ade9428939e16d89'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:31.143825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46301cc49f6d4ce9ade9428939e16d89'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:31.155880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46301cc49f6d4ce9ade9428939e16d89'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:31.156885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46301cc49f6d4ce9ade9428939e16d89'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:31.166497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1372/2756] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:09:31 2022] DELETE /identity/v3/users/46301cc49f6d4ce9ade9428939e16d89 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:31.170997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561e918-76c0-4876-995d-63dc42a9fde6 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:31.171582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561e918-76c0-4876-995d-63dc42a9fde6 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:31.171711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561e918-76c0-4876-995d-63dc42a9fde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:31.192495 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-c561e918-76c0-4876-995d-63dc42a9fde6 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:31.194235 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c561e918-76c0-4876-995d-63dc42a9fde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:31.262412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1385/2757] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:31.273450 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:31.274914 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:31.310436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:31.341637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c1e5c-d8b2-4635-ab49-f7845221a113 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:31.346099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61a931-be25-4f9e-85f1-5df62daccb48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:31.346763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61a931-be25-4f9e-85f1-5df62daccb48 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:31.347050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61a931-be25-4f9e-85f1-5df62daccb48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:31.347380 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61a931-be25-4f9e-85f1-5df62daccb48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:31.364213 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61a931-be25-4f9e-85f1-5df62daccb48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:31.366211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61a931-be25-4f9e-85f1-5df62daccb48 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:31.381032 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1373/2758] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:31.825616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561e918-76c0-4876-995d-63dc42a9fde6 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:31.828259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c561e918-76c0-4876-995d-63dc42a9fde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:31.851954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561e918-76c0-4876-995d-63dc42a9fde6 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:31.859752 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:31.860853 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:31.861140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:31.861497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] PATH_INFO: `/v3/projects/31534144acad477db46d2f0fd7c659b7` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:31.863234 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] RBAC: Authorizing `identity:delete_project(project_id=31534144acad477db46d2f0fd7c659b7)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:31.865293 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:31.938687 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31534144acad477db46d2f0fd7c659b7', 'request_id': 'req-971f1c3e-5286-4947-8ad3-b892eac7802e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:31.956425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31534144acad477db46d2f0fd7c659b7', 'request_id': 'req-971f1c3e-5286-4947-8ad3-b892eac7802e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:31.965322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31534144acad477db46d2f0fd7c659b7', 'request_id': 'req-971f1c3e-5286-4947-8ad3-b892eac7802e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:31.994632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] The token cache is being invalidate because project 31534144acad477db46d2f0fd7c659b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:09:31.995751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:32.000188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:32.005140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1386/2759] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:09:31 2022] DELETE /identity/v3/projects/31534144acad477db46d2f0fd7c659b7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:32.006705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61a931-be25-4f9e-85f1-5df62daccb48 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:32.019639 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61a931-be25-4f9e-85f1-5df62daccb48 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:32.091806 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c965d-7649-4362-81cc-674fff664044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:32.092745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c965d-7649-4362-81cc-674fff664044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:32.093382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c965d-7649-4362-81cc-674fff664044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:32.093382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c965d-7649-4362-81cc-674fff664044 admin admin] PATH_INFO: `/v3/users/3c9e117fb02f4e1c8497a01b155eaf1b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:32.094552 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c965d-7649-4362-81cc-674fff664044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c9e117fb02f4e1c8497a01b155eaf1b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:32.096767 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c965d-7649-4362-81cc-674fff664044 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:32.154042 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0b6c965d-7649-4362-81cc-674fff664044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c9e117fb02f4e1c8497a01b155eaf1b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:32.162305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0b6c965d-7649-4362-81cc-674fff664044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c9e117fb02f4e1c8497a01b155eaf1b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:32.178210 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0b6c965d-7649-4362-81cc-674fff664044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c9e117fb02f4e1c8497a01b155eaf1b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:32.187196 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0b6c965d-7649-4362-81cc-674fff664044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c9e117fb02f4e1c8497a01b155eaf1b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:32.188061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0b6c965d-7649-4362-81cc-674fff664044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c9e117fb02f4e1c8497a01b155eaf1b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:32.198526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1374/2760] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:09:32 2022] DELETE /identity/v3/users/3c9e117fb02f4e1c8497a01b155eaf1b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:32.588904 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:32.590478 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:32.597525 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:32.658133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f1c3e-5286-4947-8ad3-b892eac7802e admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:32.719965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c585c-a0d4-41be-96ff-d1bf604743a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:32.720682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c585c-a0d4-41be-96ff-d1bf604743a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:32.720975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c585c-a0d4-41be-96ff-d1bf604743a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:32.721259 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c585c-a0d4-41be-96ff-d1bf604743a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:32.723236 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-434c585c-a0d4-41be-96ff-d1bf604743a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:32.805520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c585c-a0d4-41be-96ff-d1bf604743a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:32.808469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c585c-a0d4-41be-96ff-d1bf604743a0 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:32.823555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1387/2761] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:32.841282 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78f5ea-2460-4ac5-88ca-cb1ef9ea8de5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:32.841774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78f5ea-2460-4ac5-88ca-cb1ef9ea8de5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:32.842222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78f5ea-2460-4ac5-88ca-cb1ef9ea8de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:32.871865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ef78f5ea-2460-4ac5-88ca-cb1ef9ea8de5 None None] MFA Rules not processed for user `8a7e4fed38914f91beae1c094d73163f`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:32.873127 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ef78f5ea-2460-4ac5-88ca-cb1ef9ea8de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:32.918369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:32.956886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1375/2762] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:09:32.970956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c585c-a0d4-41be-96ff-d1bf604743a0 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:32.973633 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-434c585c-a0d4-41be-96ff-d1bf604743a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:33.004652 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78f5ea-2460-4ac5-88ca-cb1ef9ea8de5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:33.006173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ef78f5ea-2460-4ac5-88ca-cb1ef9ea8de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:33.022085 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78f5ea-2460-4ac5-88ca-cb1ef9ea8de5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:33.049126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:33.050734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:33.051224 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:33.051915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] PATH_INFO: `/v3/projects/7353a84e2e924cf9a4822cdbe89b8972` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:33.053718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7353a84e2e924cf9a4822cdbe89b8972)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:33.055758 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:33.073488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c585c-a0d4-41be-96ff-d1bf604743a0 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:33.080295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981f9ad-f6aa-4f10-8200-7bd25cdd4867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:33.080873 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981f9ad-f6aa-4f10-8200-7bd25cdd4867 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:33.081252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981f9ad-f6aa-4f10-8200-7bd25cdd4867 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:33.081598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981f9ad-f6aa-4f10-8200-7bd25cdd4867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:33.089563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:33.094554 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981f9ad-f6aa-4f10-8200-7bd25cdd4867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:33.096648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981f9ad-f6aa-4f10-8200-7bd25cdd4867 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:33.107968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1388/2763] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:09:33.112233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7353a84e2e924cf9a4822cdbe89b8972', 'request_id': 'req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.119337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7353a84e2e924cf9a4822cdbe89b8972', 'request_id': 'req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.130349 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7353a84e2e924cf9a4822cdbe89b8972', 'request_id': 'req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.143268 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] The token cache is being invalidate because project 7353a84e2e924cf9a4822cdbe89b8972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:09:33.143615 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.147324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.149698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1376/2764] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:09:33 2022] DELETE /identity/v3/projects/7353a84e2e924cf9a4822cdbe89b8972 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:09:33.153899 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981f9ad-f6aa-4f10-8200-7bd25cdd4867 service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:33.168498 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981f9ad-f6aa-4f10-8200-7bd25cdd4867 service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:33.237113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:33.238127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:33.239066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:33.239431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] PATH_INFO: `/v3/users/4696ac2517e44acaaee9f3a577c4b652` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:33.240812 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4696ac2517e44acaaee9f3a577c4b652)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:33.242862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:33.302476 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4696ac2517e44acaaee9f3a577c4b652'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.313673 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4696ac2517e44acaaee9f3a577c4b652'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.314625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4696ac2517e44acaaee9f3a577c4b652'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.325238 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4696ac2517e44acaaee9f3a577c4b652'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.325982 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4696ac2517e44acaaee9f3a577c4b652'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.358607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1389/2765] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:09:33 2022] DELETE /identity/v3/users/4696ac2517e44acaaee9f3a577c4b652 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:33.481646 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:33.483320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:33.502944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c08d1a-e3e7-46bf-bbd2-5f610ae26bc9 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:33.570959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb03a58-ca92-453b-9161-08f07a3aabd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:33.571847 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb03a58-ca92-453b-9161-08f07a3aabd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:33.572136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb03a58-ca92-453b-9161-08f07a3aabd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:33.572410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb03a58-ca92-453b-9161-08f07a3aabd8 admin admin] PATH_INFO: `/v3/users/b6a2a022db8b4ef98a8c4932f753240a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:33.574308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb03a58-ca92-453b-9161-08f07a3aabd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6a2a022db8b4ef98a8c4932f753240a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:33.576262 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb03a58-ca92-453b-9161-08f07a3aabd8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:33.646401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ccb03a58-ca92-453b-9161-08f07a3aabd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6a2a022db8b4ef98a8c4932f753240a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.655575 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ccb03a58-ca92-453b-9161-08f07a3aabd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6a2a022db8b4ef98a8c4932f753240a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.664016 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:33.671993 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ccb03a58-ca92-453b-9161-08f07a3aabd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6a2a022db8b4ef98a8c4932f753240a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.687078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ccb03a58-ca92-453b-9161-08f07a3aabd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6a2a022db8b4ef98a8c4932f753240a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.688051 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-ccb03a58-ca92-453b-9161-08f07a3aabd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6a2a022db8b4ef98a8c4932f753240a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.696666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1377/2766] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:09:33 2022] DELETE /identity/v3/users/b6a2a022db8b4ef98a8c4932f753240a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:33.799278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:33.800841 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:33.826459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60076c96-946e-48c0-b5a9-7f2ddb8ecf82 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:33.911371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:33.913469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:33.913937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:33.914505 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] PATH_INFO: `/v3/projects/b719384d2b9746f88bc816b8b90987c2` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:33.915864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b719384d2b9746f88bc816b8b90987c2)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:33.918143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:33.938642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:33.977008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b719384d2b9746f88bc816b8b90987c2', 'request_id': 'req-0a20c40c-3c57-4688-88f0-4f8722b74884'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.989297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b719384d2b9746f88bc816b8b90987c2', 'request_id': 'req-0a20c40c-3c57-4688-88f0-4f8722b74884'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:33.998745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b719384d2b9746f88bc816b8b90987c2', 'request_id': 'req-0a20c40c-3c57-4688-88f0-4f8722b74884'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:34.011846 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] The token cache is being invalidate because project b719384d2b9746f88bc816b8b90987c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:09:34.012443 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:34.013343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:34.019985 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1390/2767] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:09:33 2022] DELETE /identity/v3/projects/b719384d2b9746f88bc816b8b90987c2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:34.021781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818e43e-e464-4348-a27e-3dd756698cd4 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:34.022068 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818e43e-e464-4348-a27e-3dd756698cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:34.022340 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818e43e-e464-4348-a27e-3dd756698cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:34.043228 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8818e43e-e464-4348-a27e-3dd756698cd4 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:34.045565 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8818e43e-e464-4348-a27e-3dd756698cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:34.128202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:34.132894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:34.134592 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1378/2768] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:34.143259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e356c64-405a-4d9a-b510-512b2bffb070 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:34.148616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e356c64-405a-4d9a-b510-512b2bffb070 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:34.148926 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e356c64-405a-4d9a-b510-512b2bffb070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:34.150629 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20c40c-3c57-4688-88f0-4f8722b74884 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:34.156939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:34.158065 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:34.158488 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:34.158872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] PATH_INFO: `/v3/projects/b8abe8948c934248a448532711d4499b` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:34.160503 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8abe8948c934248a448532711d4499b)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:34.162582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:34.195885 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2e356c64-405a-4d9a-b510-512b2bffb070 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:34.197817 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2e356c64-405a-4d9a-b510-512b2bffb070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:34.213429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1379/2769] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:34.216155 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e356c64-405a-4d9a-b510-512b2bffb070 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:34.226060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e356c64-405a-4d9a-b510-512b2bffb070 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:34.236628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8abe8948c934248a448532711d4499b', 'request_id': 'req-0879f49e-82b2-48c9-8030-d997499d2673'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:34.251642 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8abe8948c934248a448532711d4499b', 'request_id': 'req-0879f49e-82b2-48c9-8030-d997499d2673'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:34.261206 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8abe8948c934248a448532711d4499b', 'request_id': 'req-0879f49e-82b2-48c9-8030-d997499d2673'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:34.278685 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] The token cache is being invalidate because project b8abe8948c934248a448532711d4499b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:09:34.279768 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:34.282121 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:34.285961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1391/2770] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:09:34 2022] DELETE /identity/v3/projects/b8abe8948c934248a448532711d4499b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:34.297837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89a827-548c-432e-a6f9-57125db5d680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:34.298531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89a827-548c-432e-a6f9-57125db5d680 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:34.299042 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89a827-548c-432e-a6f9-57125db5d680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:34.299323 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89a827-548c-432e-a6f9-57125db5d680 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:34.300090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89a827-548c-432e-a6f9-57125db5d680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:09:34.300598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89a827-548c-432e-a6f9-57125db5d680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:34.302512 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89a827-548c-432e-a6f9-57125db5d680 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:34.324433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1380/2771] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:09:34.325247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:34.326763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:34.344834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879f49e-82b2-48c9-8030-d997499d2673 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:34.350465 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9be544-9fef-477f-9bf4-d4fc3e18f95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:34.351292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9be544-9fef-477f-9bf4-d4fc3e18f95e admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:34.351730 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9be544-9fef-477f-9bf4-d4fc3e18f95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:34.352014 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9be544-9fef-477f-9bf4-d4fc3e18f95e admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:34.353132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9be544-9fef-477f-9bf4-d4fc3e18f95e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:34.357529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9be544-9fef-477f-9bf4-d4fc3e18f95e admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:34.412933 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1392/2772] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:09:34 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:09:34.422921 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89a827-548c-432e-a6f9-57125db5d680 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:34.450185 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89a827-548c-432e-a6f9-57125db5d680 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:34.530781 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886849e-657a-4937-b34c-fae32d3aeeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:34.531735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886849e-657a-4937-b34c-fae32d3aeeef admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:34.532119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886849e-657a-4937-b34c-fae32d3aeeef admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:34.532610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886849e-657a-4937-b34c-fae32d3aeeef admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:34.533422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886849e-657a-4937-b34c-fae32d3aeeef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:34.535525 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886849e-657a-4937-b34c-fae32d3aeeef admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:34.553784 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:34.563006 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1381/2773] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:09:34 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:09:34.564182 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9be544-9fef-477f-9bf4-d4fc3e18f95e admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:34.575295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9be544-9fef-477f-9bf4-d4fc3e18f95e admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:34.580773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bde9d-e638-4d1f-af02-e0f2720e5816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:34.581482 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bde9d-e638-4d1f-af02-e0f2720e5816 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:34.582375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bde9d-e638-4d1f-af02-e0f2720e5816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:34.582375 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bde9d-e638-4d1f-af02-e0f2720e5816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:34.583048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153bde9d-e638-4d1f-af02-e0f2720e5816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:34.584667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153bde9d-e638-4d1f-af02-e0f2720e5816 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:34.596469 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1393/2774] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:34.600434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886849e-657a-4937-b34c-fae32d3aeeef admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:34.612066 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886849e-657a-4937-b34c-fae32d3aeeef admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:34.617028 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007794a-dbc9-4ab3-afde-e618a7e5fa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:34.617681 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007794a-dbc9-4ab3-afde-e618a7e5fa49 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:34.618105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007794a-dbc9-4ab3-afde-e618a7e5fa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:34.618796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007794a-dbc9-4ab3-afde-e618a7e5fa49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:34.619243 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007794a-dbc9-4ab3-afde-e618a7e5fa49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:34.621947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007794a-dbc9-4ab3-afde-e618a7e5fa49 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:34.634043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1382/2775] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:34.639493 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bde9d-e638-4d1f-af02-e0f2720e5816 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:34.651497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bde9d-e638-4d1f-af02-e0f2720e5816 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:34.658316 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e360fc-adf5-4ddc-b57a-fd35bb2ebc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:34.659348 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e360fc-adf5-4ddc-b57a-fd35bb2ebc44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:34.659674 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e360fc-adf5-4ddc-b57a-fd35bb2ebc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:34.659952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e360fc-adf5-4ddc-b57a-fd35bb2ebc44 admin admin] PATH_INFO: `/v3/projects/19d7982e502a432da43d049b5bde2888/users/c114dd46ee13454f90d6e91eba795751/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:34.678503 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e360fc-adf5-4ddc-b57a-fd35bb2ebc44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d7982e502a432da43d049b5bde2888, user_id=c114dd46ee13454f90d6e91eba795751, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:34.682024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e360fc-adf5-4ddc-b57a-fd35bb2ebc44 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:34.708825 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1394/2776] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:34 2022] PUT /identity/v3/projects/19d7982e502a432da43d049b5bde2888/users/c114dd46ee13454f90d6e91eba795751/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:34.712904 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f9b0f-4bb0-49cf-92ac-10e677995114 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:34.713401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f9b0f-4bb0-49cf-92ac-10e677995114 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:34.714024 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f9b0f-4bb0-49cf-92ac-10e677995114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:34.743878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a24f9b0f-4bb0-49cf-92ac-10e677995114 None None] MFA Rules not processed for user `c114dd46ee13454f90d6e91eba795751`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:34.746737 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a24f9b0f-4bb0-49cf-92ac-10e677995114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:34.852574 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f2daf-7d78-496f-b063-4707a50f1460 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:34.853097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f2daf-7d78-496f-b063-4707a50f1460 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:34.853204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f2daf-7d78-496f-b063-4707a50f1460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:34.856440 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1383/2777] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:09:34.868843 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f9b0f-4bb0-49cf-92ac-10e677995114 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:34.871766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a24f9b0f-4bb0-49cf-92ac-10e677995114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:34.877148 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-811f2daf-7d78-496f-b063-4707a50f1460 None None] MFA Rules not processed for user `8a7e4fed38914f91beae1c094d73163f`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:34.878366 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-811f2daf-7d78-496f-b063-4707a50f1460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:34.907539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:34.938035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f9b0f-4bb0-49cf-92ac-10e677995114 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:34.941812 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e7993-aa71-4fd2-8e30-5ab64cb2f441 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:34.942660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e7993-aa71-4fd2-8e30-5ab64cb2f441 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:34.942826 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e7993-aa71-4fd2-8e30-5ab64cb2f441 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:34.943146 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e7993-aa71-4fd2-8e30-5ab64cb2f441 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:34.943956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1395/2778] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:34.954913 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e7993-aa71-4fd2-8e30-5ab64cb2f441 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:34.956741 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e7993-aa71-4fd2-8e30-5ab64cb2f441 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:34.969253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1384/2779] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:09:35.313611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f2daf-7d78-496f-b063-4707a50f1460 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:35.315377 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-811f2daf-7d78-496f-b063-4707a50f1460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:35.409122 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f2daf-7d78-496f-b063-4707a50f1460 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:35.417910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b09f5a-2579-4958-9466-0911cb67b6f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:35.417910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b09f5a-2579-4958-9466-0911cb67b6f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:35.417910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b09f5a-2579-4958-9466-0911cb67b6f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:35.417910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b09f5a-2579-4958-9466-0911cb67b6f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:35.418570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c8b09f5a-2579-4958-9466-0911cb67b6f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:35.443164 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b09f5a-2579-4958-9466-0911cb67b6f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:35.445038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b09f5a-2579-4958-9466-0911cb67b6f8 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:35.456439 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1396/2780] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:35.834792 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e7993-aa71-4fd2-8e30-5ab64cb2f441 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:35.836526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8a0e7993-aa71-4fd2-8e30-5ab64cb2f441 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:35.963957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e7993-aa71-4fd2-8e30-5ab64cb2f441 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:35.969451 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9b21f-f63a-4a9d-8193-8dccd81b7c53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:35.970115 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9b21f-f63a-4a9d-8193-8dccd81b7c53 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:35.970401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9b21f-f63a-4a9d-8193-8dccd81b7c53 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:35.970677 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9b21f-f63a-4a9d-8193-8dccd81b7c53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:35.972563 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7cd9b21f-f63a-4a9d-8193-8dccd81b7c53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:35.987326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9b21f-f63a-4a9d-8193-8dccd81b7c53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:35.989039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9b21f-f63a-4a9d-8193-8dccd81b7c53 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:36.004187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1385/2781] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:36.493745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2d707-8338-48f3-80fe-4c89d40590a9 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:36.495290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2d707-8338-48f3-80fe-4c89d40590a9 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:36.495290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2d707-8338-48f3-80fe-4c89d40590a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:36.517685 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-4ae2d707-8338-48f3-80fe-4c89d40590a9 None None] MFA Rules not processed for user `c114dd46ee13454f90d6e91eba795751`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:36.519188 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4ae2d707-8338-48f3-80fe-4c89d40590a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:36.527078 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1397/2782] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:40.022549 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9b21f-f63a-4a9d-8193-8dccd81b7c53 service placement] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:40.024060 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7cd9b21f-f63a-4a9d-8193-8dccd81b7c53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:40.056095 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:40.122581 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9b21f-f63a-4a9d-8193-8dccd81b7c53 service placement] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:40.127995 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1868d-25b0-4d20-821a-5a73e6abf8ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:40.128679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1868d-25b0-4d20-821a-5a73e6abf8ab service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:40.128963 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1868d-25b0-4d20-821a-5a73e6abf8ab service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:40.129244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1868d-25b0-4d20-821a-5a73e6abf8ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:40.131050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-06f1868d-25b0-4d20-821a-5a73e6abf8ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:40.144401 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1868d-25b0-4d20-821a-5a73e6abf8ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:40.146441 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1868d-25b0-4d20-821a-5a73e6abf8ab service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:40.158116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1386/2783] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:46.108905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2d707-8338-48f3-80fe-4c89d40590a9 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:46.110648 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4ae2d707-8338-48f3-80fe-4c89d40590a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:46.129739 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2d707-8338-48f3-80fe-4c89d40590a9 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:46.136096 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e476f-1358-4c51-ae88-df3b3d371642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:46.136778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e476f-1358-4c51-ae88-df3b3d371642 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:46.137062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e476f-1358-4c51-ae88-df3b3d371642 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:46.137333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e476f-1358-4c51-ae88-df3b3d371642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:46.139090 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-272e476f-1358-4c51-ae88-df3b3d371642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:46.151907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e476f-1358-4c51-ae88-df3b3d371642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:46.153500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e476f-1358-4c51-ae88-df3b3d371642 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:46.164709 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1398/2784] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:09:50.153647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=29 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:54.069783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8727d71-339f-4fe7-8ef5-e93baa3ce7e3 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:54.070354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8727d71-339f-4fe7-8ef5-e93baa3ce7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:54.070354 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8727d71-339f-4fe7-8ef5-e93baa3ce7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:54.130613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b8727d71-339f-4fe7-8ef5-e93baa3ce7e3 None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:54.131810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b8727d71-339f-4fe7-8ef5-e93baa3ce7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:54.193896 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1387/2785] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 23:09:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:54.240297 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e476f-1358-4c51-ae88-df3b3d371642 service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:54.241740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-272e476f-1358-4c51-ae88-df3b3d371642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:54.332243 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e476f-1358-4c51-ae88-df3b3d371642 service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:54.337287 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dc558-0abf-4929-8f4c-0f96bc3dcd31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:54.338750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dc558-0abf-4929-8f4c-0f96bc3dcd31 service swift] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:54.339036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dc558-0abf-4929-8f4c-0f96bc3dcd31 service swift] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:54.339335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dc558-0abf-4929-8f4c-0f96bc3dcd31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:54.347184 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dc558-0abf-4929-8f4c-0f96bc3dcd31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:54.348784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dc558-0abf-4929-8f4c-0f96bc3dcd31 service swift] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:54.356813 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1399/2786] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 23:09:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 23:09:54.879429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c80a13-4424-4920-bd9b-874d5ec23932 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:54.879723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c80a13-4424-4920-bd9b-874d5ec23932 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:54.880027 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c80a13-4424-4920-bd9b-874d5ec23932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:54.898147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c4c80a13-4424-4920-bd9b-874d5ec23932 None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:54.899346 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c4c80a13-4424-4920-bd9b-874d5ec23932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:54.907763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1388/2787] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jul 21 23:09:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:54.920279 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dc558-0abf-4929-8f4c-0f96bc3dcd31 service swift] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:54.931611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dc558-0abf-4929-8f4c-0f96bc3dcd31 service swift] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:54.936721 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d1c23-26f5-49e0-9da6-05327c7e19d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:54.937363 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1c23-26f5-49e0-9da6-05327c7e19d0 service swift] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:54.937651 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1c23-26f5-49e0-9da6-05327c7e19d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:54.937934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1c23-26f5-49e0-9da6-05327c7e19d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:54.946030 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d1c23-26f5-49e0-9da6-05327c7e19d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:54.948426 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d1c23-26f5-49e0-9da6-05327c7e19d0 service swift] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:54.956523 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1400/2788] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 23:09:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 23:09:55.143900 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014250c-7dc2-4c06-bbcd-c139f5f6242a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:55.144376 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014250c-7dc2-4c06-bbcd-c139f5f6242a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:55.144534 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014250c-7dc2-4c06-bbcd-c139f5f6242a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:55.161618 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0014250c-7dc2-4c06-bbcd-c139f5f6242a None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:55.162791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0014250c-7dc2-4c06-bbcd-c139f5f6242a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:55.170436 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1389/2789] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 23:09:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:09:55.183173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d1c23-26f5-49e0-9da6-05327c7e19d0 service swift] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:55.193675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d1c23-26f5-49e0-9da6-05327c7e19d0 service swift] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:55.199520 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e867710-3900-4398-b580-79c4d78f9583 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:55.200156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e867710-3900-4398-b580-79c4d78f9583 service swift] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:55.200442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e867710-3900-4398-b580-79c4d78f9583 service swift] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:55.200778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e867710-3900-4398-b580-79c4d78f9583 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:55.209017 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e867710-3900-4398-b580-79c4d78f9583 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:55.210570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e867710-3900-4398-b580-79c4d78f9583 service swift] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:55.218970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1401/2790] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 23:09:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 23:09:55.531572 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a6934-1a3c-4ad6-85a7-84bd4ce689c0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:55.531861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a6934-1a3c-4ad6-85a7-84bd4ce689c0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:55.532141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a6934-1a3c-4ad6-85a7-84bd4ce689c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:55.550034 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-499a6934-1a3c-4ad6-85a7-84bd4ce689c0 None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:55.551221 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-499a6934-1a3c-4ad6-85a7-84bd4ce689c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:55.558428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1390/2791] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jul 21 23:09:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:55.570165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e867710-3900-4398-b580-79c4d78f9583 service swift] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:55.575483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=100 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:55.581881 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e867710-3900-4398-b580-79c4d78f9583 service swift] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:55.587032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ebee3-677c-4de6-abdc-f34f08aff732 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:55.588278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ebee3-677c-4de6-abdc-f34f08aff732 service swift] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:55.588454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ebee3-677c-4de6-abdc-f34f08aff732 service swift] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:55.588771 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ebee3-677c-4de6-abdc-f34f08aff732 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:55.596942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ebee3-677c-4de6-abdc-f34f08aff732 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:55.598541 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ebee3-677c-4de6-abdc-f34f08aff732 service swift] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:55.607465 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1402/2792] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 23:09:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 23:09:57.652553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a6934-1a3c-4ad6-85a7-84bd4ce689c0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:57.715033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a6934-1a3c-4ad6-85a7-84bd4ce689c0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:57.719403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:57.720257 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:57.720564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:57.720839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] PATH_INFO: `/v3/users/c114dd46ee13454f90d6e91eba795751` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:57.722033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c114dd46ee13454f90d6e91eba795751)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:57.723936 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:57.776506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:57.778314 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c114dd46ee13454f90d6e91eba795751'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:57.787074 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c114dd46ee13454f90d6e91eba795751'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:57.800087 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c114dd46ee13454f90d6e91eba795751'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:57.808339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c114dd46ee13454f90d6e91eba795751'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:57.808990 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c114dd46ee13454f90d6e91eba795751'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:57.815140 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1391/2793] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:09:57 2022] DELETE /identity/v3/users/c114dd46ee13454f90d6e91eba795751 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:57.819560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14bf6c-745f-4a54-82bf-356ae8a383df None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:57.820062 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14bf6c-745f-4a54-82bf-356ae8a383df None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:57.820185 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14bf6c-745f-4a54-82bf-356ae8a383df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:57.838091 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-2e14bf6c-745f-4a54-82bf-356ae8a383df None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:57.839693 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2e14bf6c-745f-4a54-82bf-356ae8a383df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:57.900837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1403/2794] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:57.910715 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:57.912118 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:57.970255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a0f6d-1e8d-47ac-861b-6a0e5fc06398 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:57.973810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bdd25-e8c2-4843-a66b-1739d3721c7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:57.974447 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bdd25-e8c2-4843-a66b-1739d3721c7a service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:57.974730 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bdd25-e8c2-4843-a66b-1739d3721c7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:57.975003 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bdd25-e8c2-4843-a66b-1739d3721c7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:57.983968 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bdd25-e8c2-4843-a66b-1739d3721c7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:57.985433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bdd25-e8c2-4843-a66b-1739d3721c7a service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:57.995188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1392/2795] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:09:58.164139 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14bf6c-745f-4a54-82bf-356ae8a383df None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:58.165827 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2e14bf6c-745f-4a54-82bf-356ae8a383df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:58.178339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14bf6c-745f-4a54-82bf-356ae8a383df None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:58.184509 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63ed5e-5951-4ff9-9999-5df655a52057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:58.185518 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63ed5e-5951-4ff9-9999-5df655a52057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:58.185921 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63ed5e-5951-4ff9-9999-5df655a52057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:58.186320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63ed5e-5951-4ff9-9999-5df655a52057 admin admin] PATH_INFO: `/v3/projects/19d7982e502a432da43d049b5bde2888` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:58.187698 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63ed5e-5951-4ff9-9999-5df655a52057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d7982e502a432da43d049b5bde2888)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:58.189716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63ed5e-5951-4ff9-9999-5df655a52057 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:58.237514 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7d63ed5e-5951-4ff9-9999-5df655a52057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d7982e502a432da43d049b5bde2888', 'request_id': 'req-7d63ed5e-5951-4ff9-9999-5df655a52057'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:58.244888 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7d63ed5e-5951-4ff9-9999-5df655a52057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d7982e502a432da43d049b5bde2888', 'request_id': 'req-7d63ed5e-5951-4ff9-9999-5df655a52057'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:58.250358 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7d63ed5e-5951-4ff9-9999-5df655a52057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d7982e502a432da43d049b5bde2888', 'request_id': 'req-7d63ed5e-5951-4ff9-9999-5df655a52057'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:58.261522 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7d63ed5e-5951-4ff9-9999-5df655a52057 admin admin] The token cache is being invalidate because project 19d7982e502a432da43d049b5bde2888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:09:58.261810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7d63ed5e-5951-4ff9-9999-5df655a52057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:58.262442 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7d63ed5e-5951-4ff9-9999-5df655a52057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:09:58.263974 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1404/2796] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:09:58 2022] DELETE /identity/v3/projects/19d7982e502a432da43d049b5bde2888 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:09:58.268859 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354078d-a0c8-48ee-be0d-95100ca20feb None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:58.269149 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354078d-a0c8-48ee-be0d-95100ca20feb None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:58.269424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354078d-a0c8-48ee-be0d-95100ca20feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:58.287287 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4354078d-a0c8-48ee-be0d-95100ca20feb None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:58.288521 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4354078d-a0c8-48ee-be0d-95100ca20feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:58.347863 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1393/2797] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:58.356712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48d3c8-3978-42fa-a1f6-7c1c1c0f04d9 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:58.356999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48d3c8-3978-42fa-a1f6-7c1c1c0f04d9 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:58.357284 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48d3c8-3978-42fa-a1f6-7c1c1c0f04d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:58.375763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-4b48d3c8-3978-42fa-a1f6-7c1c1c0f04d9 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:58.377011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4b48d3c8-3978-42fa-a1f6-7c1c1c0f04d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:58.385984 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1405/2798] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:58.390043 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354078d-a0c8-48ee-be0d-95100ca20feb None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:58.400827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354078d-a0c8-48ee-be0d-95100ca20feb None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:58.405174 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da686dc7-1240-4e35-8975-9deec2858095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:58.405822 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da686dc7-1240-4e35-8975-9deec2858095 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:58.406101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da686dc7-1240-4e35-8975-9deec2858095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:58.406374 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da686dc7-1240-4e35-8975-9deec2858095 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:58.407099 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da686dc7-1240-4e35-8975-9deec2858095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:09:58.407403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da686dc7-1240-4e35-8975-9deec2858095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:58.409036 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da686dc7-1240-4e35-8975-9deec2858095 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:58.421759 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1394/2799] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:09:58.425964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48d3c8-3978-42fa-a1f6-7c1c1c0f04d9 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:58.435454 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48d3c8-3978-42fa-a1f6-7c1c1c0f04d9 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:58.440284 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63769a-563c-401e-9c36-8d14333e2515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:58.440986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63769a-563c-401e-9c36-8d14333e2515 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:58.441276 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63769a-563c-401e-9c36-8d14333e2515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:58.441555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63769a-563c-401e-9c36-8d14333e2515 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:58.442265 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63769a-563c-401e-9c36-8d14333e2515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:58.444225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63769a-563c-401e-9c36-8d14333e2515 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:58.470923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1406/2800] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:09:58 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:09:58.475129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da686dc7-1240-4e35-8975-9deec2858095 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:58.486179 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da686dc7-1240-4e35-8975-9deec2858095 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:58.534645 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:58.546413 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3403e-02ce-4e71-9beb-d35f6cad781e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:58.547126 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3403e-02ce-4e71-9beb-d35f6cad781e admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:58.547588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3403e-02ce-4e71-9beb-d35f6cad781e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:58.547868 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3403e-02ce-4e71-9beb-d35f6cad781e admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:58.548595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3403e-02ce-4e71-9beb-d35f6cad781e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:58.550521 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3403e-02ce-4e71-9beb-d35f6cad781e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:58.570655 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1395/2801] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:09:58 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:09:58.575542 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63769a-563c-401e-9c36-8d14333e2515 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:58.585735 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63769a-563c-401e-9c36-8d14333e2515 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:58.591039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3eb4c-295b-40bd-b159-1c2ad3664dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:58.591707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3eb4c-295b-40bd-b159-1c2ad3664dad admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:58.591993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3eb4c-295b-40bd-b159-1c2ad3664dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:58.592273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3eb4c-295b-40bd-b159-1c2ad3664dad admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:58.593045 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3eb4c-295b-40bd-b159-1c2ad3664dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:58.594636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3eb4c-295b-40bd-b159-1c2ad3664dad admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:58.606593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1407/2802] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:09:58.610490 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3403e-02ce-4e71-9beb-d35f6cad781e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:58.620718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3403e-02ce-4e71-9beb-d35f6cad781e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:58.625333 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946162c8-0a9f-46a2-8968-ed7fb4431bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:58.625989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946162c8-0a9f-46a2-8968-ed7fb4431bed admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:58.626269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946162c8-0a9f-46a2-8968-ed7fb4431bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:58.626548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946162c8-0a9f-46a2-8968-ed7fb4431bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:58.627274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946162c8-0a9f-46a2-8968-ed7fb4431bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:58.628912 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946162c8-0a9f-46a2-8968-ed7fb4431bed admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:58.638943 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1396/2803] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:09:58.642849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3eb4c-295b-40bd-b159-1c2ad3664dad admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:58.653628 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3eb4c-295b-40bd-b159-1c2ad3664dad admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:58.658230 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ae232-f4e6-4227-b6e5-4d2dad2b7b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:58.659113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ae232-f4e6-4227-b6e5-4d2dad2b7b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:58.659402 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ae232-f4e6-4227-b6e5-4d2dad2b7b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:58.659682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ae232-f4e6-4227-b6e5-4d2dad2b7b9b admin admin] PATH_INFO: `/v3/projects/78066cfe2b844adfb3b6c550f662f4fa/users/07007b0e57ed4dcbb679d813f96321cb/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:58.674390 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ae232-f4e6-4227-b6e5-4d2dad2b7b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=78066cfe2b844adfb3b6c550f662f4fa, user_id=07007b0e57ed4dcbb679d813f96321cb, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:58.677041 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ae232-f4e6-4227-b6e5-4d2dad2b7b9b admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:58.690097 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1408/2804] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:09:58 2022] PUT /identity/v3/projects/78066cfe2b844adfb3b6c550f662f4fa/users/07007b0e57ed4dcbb679d813f96321cb/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:09:58.700666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebde8a-979e-4ecb-87ff-ce3e70d1df40 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:58.701186 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebde8a-979e-4ecb-87ff-ce3e70d1df40 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:58.701322 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebde8a-979e-4ecb-87ff-ce3e70d1df40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:58.727431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-33ebde8a-979e-4ecb-87ff-ce3e70d1df40 None None] MFA Rules not processed for user `07007b0e57ed4dcbb679d813f96321cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:09:58.728763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-33ebde8a-979e-4ecb-87ff-ce3e70d1df40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:58.790233 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1397/2805] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:09:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:09:58.801246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ae232-f4e6-4227-b6e5-4d2dad2b7b9b admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:09:58.802943 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-643ae232-f4e6-4227-b6e5-4d2dad2b7b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:09:58.814560 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:09:58.861302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ae232-f4e6-4227-b6e5-4d2dad2b7b9b admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:09:58.864939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeff7f6-a5d3-4789-98e6-a050dbe3eb1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:09:58.865587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeff7f6-a5d3-4789-98e6-a050dbe3eb1f service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:09:58.865872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeff7f6-a5d3-4789-98e6-a050dbe3eb1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:09:58.866150 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeff7f6-a5d3-4789-98e6-a050dbe3eb1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:09:58.875342 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeff7f6-a5d3-4789-98e6-a050dbe3eb1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:09:58.876793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeff7f6-a5d3-4789-98e6-a050dbe3eb1f service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:09:58.885491 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1409/2806] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:09:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:10:08.787260 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:10:08.881455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=22 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:10:18.094180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebde8a-979e-4ecb-87ff-ce3e70d1df40 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:10:18.157259 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebde8a-979e-4ecb-87ff-ce3e70d1df40 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:10:18.161811 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:10:18.162676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:10:18.162955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:10:18.163226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] PATH_INFO: `/v3/users/07007b0e57ed4dcbb679d813f96321cb` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:10:18.164502 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] RBAC: Authorizing `identity:delete_user(user_id=07007b0e57ed4dcbb679d813f96321cb)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:10:18.166452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:10:18.210952 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07007b0e57ed4dcbb679d813f96321cb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:10:18.218414 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07007b0e57ed4dcbb679d813f96321cb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:10:18.230211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07007b0e57ed4dcbb679d813f96321cb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:10:18.237903 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07007b0e57ed4dcbb679d813f96321cb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:10:18.238556 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07007b0e57ed4dcbb679d813f96321cb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:10:18.244777 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1398/2807] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:10:18 2022] DELETE /identity/v3/users/07007b0e57ed4dcbb679d813f96321cb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:10:18.249363 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18939bce-a19b-4435-8aa8-79b72122f2cb None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:10:18.249851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18939bce-a19b-4435-8aa8-79b72122f2cb None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:10:18.249980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18939bce-a19b-4435-8aa8-79b72122f2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:10:18.267350 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-18939bce-a19b-4435-8aa8-79b72122f2cb None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:10:18.268676 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-18939bce-a19b-4435-8aa8-79b72122f2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:10:18.327380 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1410/2808] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:10:18.337477 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:10:18.339508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:10:18.398555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ceaeb-bd8d-4a0f-a748-9ecd1df0d2be admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:10:18.402911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e92c7a-a6b8-4787-adae-6434e14d81a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:10:18.403572 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e92c7a-a6b8-4787-adae-6434e14d81a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:10:18.403858 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e92c7a-a6b8-4787-adae-6434e14d81a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:10:18.404139 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e92c7a-a6b8-4787-adae-6434e14d81a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:10:18.413584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e92c7a-a6b8-4787-adae-6434e14d81a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:10:18.415044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e92c7a-a6b8-4787-adae-6434e14d81a9 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:10:18.425647 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1399/2809] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:10:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:10:18.687289 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18939bce-a19b-4435-8aa8-79b72122f2cb None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:10:18.688736 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-18939bce-a19b-4435-8aa8-79b72122f2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:10:18.701764 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18939bce-a19b-4435-8aa8-79b72122f2cb None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:10:18.706386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:10:18.707286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:10:18.707622 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:10:18.707907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] PATH_INFO: `/v3/projects/78066cfe2b844adfb3b6c550f662f4fa` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:10:18.709202 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78066cfe2b844adfb3b6c550f662f4fa)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:10:18.711081 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:10:18.756941 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78066cfe2b844adfb3b6c550f662f4fa', 'request_id': 'req-7c101542-756e-4765-8d00-19f80a4972f7'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:10:18.765052 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78066cfe2b844adfb3b6c550f662f4fa', 'request_id': 'req-7c101542-756e-4765-8d00-19f80a4972f7'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:10:18.771038 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78066cfe2b844adfb3b6c550f662f4fa', 'request_id': 'req-7c101542-756e-4765-8d00-19f80a4972f7'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:10:18.781927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] The token cache is being invalidate because project 78066cfe2b844adfb3b6c550f662f4fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:10:18.782211 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:10:18.782862 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:10:18.784372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1411/2810] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:10:18 2022] DELETE /identity/v3/projects/78066cfe2b844adfb3b6c550f662f4fa => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:10:28.421166 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:10:28.779058 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:11:44.578785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e92c7a-a6b8-4787-adae-6434e14d81a9 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:11:44.587959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e92c7a-a6b8-4787-adae-6434e14d81a9 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:11:44.637075 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e9ac6-f426-4744-80c8-022f62ee3c8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:11:44.637716 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e9ac6-f426-4744-80c8-022f62ee3c8a service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:11:44.637999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e9ac6-f426-4744-80c8-022f62ee3c8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:11:44.638284 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e9ac6-f426-4744-80c8-022f62ee3c8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:11:44.640050 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c40e9ac6-f426-4744-80c8-022f62ee3c8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:11:44.708592 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e9ac6-f426-4744-80c8-022f62ee3c8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:11:44.710107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e9ac6-f426-4744-80c8-022f62ee3c8a service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:11:44.721186 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1400/2811] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:11:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:11:54.716358 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=54 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:12:16.198239 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:12:16.199682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:12:16.212396 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c101542-756e-4765-8d00-19f80a4972f7 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:12:16.215861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9fb57-e65e-4f4d-9d81-c4a5ff44e85d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:12:16.216528 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9fb57-e65e-4f4d-9d81-c4a5ff44e85d service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:12:16.216817 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9fb57-e65e-4f4d-9d81-c4a5ff44e85d service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:12:16.217094 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9fb57-e65e-4f4d-9d81-c4a5ff44e85d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:12:16.218816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-99b9fb57-e65e-4f4d-9d81-c4a5ff44e85d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:12:16.228743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9fb57-e65e-4f4d-9d81-c4a5ff44e85d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:12:16.230161 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9fb57-e65e-4f4d-9d81-c4a5ff44e85d service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:12:16.238123 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1412/2812] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:12:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:12:17.839715 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e9ac6-f426-4744-80c8-022f62ee3c8a service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:12:17.848598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e9ac6-f426-4744-80c8-022f62ee3c8a service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:12:17.853251 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cde9a-7351-4a8f-809c-6e371aac3e06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:12:17.854892 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cde9a-7351-4a8f-809c-6e371aac3e06 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:12:17.855394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cde9a-7351-4a8f-809c-6e371aac3e06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:12:17.855678 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cde9a-7351-4a8f-809c-6e371aac3e06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:12:17.858111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-102cde9a-7351-4a8f-809c-6e371aac3e06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:12:17.873035 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102cde9a-7351-4a8f-809c-6e371aac3e06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:12:17.874518 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102cde9a-7351-4a8f-809c-6e371aac3e06 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:12:17.885955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1401/2813] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:12:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:12:24.679849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fafaf1-dd68-47d5-9059-be7ca017fa09 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:12:24.680353 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fafaf1-dd68-47d5-9059-be7ca017fa09 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:12:24.680459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fafaf1-dd68-47d5-9059-be7ca017fa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:12:24.697934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-73fafaf1-dd68-47d5-9059-be7ca017fa09 None None] MFA Rules not processed for user `8a7e4fed38914f91beae1c094d73163f`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:12:24.699059 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-73fafaf1-dd68-47d5-9059-be7ca017fa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:12:24.752544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1413/2814] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:12:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:12:27.880839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=6 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:12:34.746152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=31 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:13:19.572147 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cde9a-7351-4a8f-809c-6e371aac3e06 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:13:19.573575 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-102cde9a-7351-4a8f-809c-6e371aac3e06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:13:19.680103 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cde9a-7351-4a8f-809c-6e371aac3e06 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:13:19.685275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98694fd5-98a9-4105-8b52-61891886cf13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:13:19.685860 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98694fd5-98a9-4105-8b52-61891886cf13 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:19.686145 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98694fd5-98a9-4105-8b52-61891886cf13 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:19.686417 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98694fd5-98a9-4105-8b52-61891886cf13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:19.688188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-98694fd5-98a9-4105-8b52-61891886cf13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:13:19.700068 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98694fd5-98a9-4105-8b52-61891886cf13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:13:19.701719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98694fd5-98a9-4105-8b52-61891886cf13 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:13:19.711564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1402/2815] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:13:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:13:29.707324 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=49 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:13:47.490036 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fafaf1-dd68-47d5-9059-be7ca017fa09 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:13:47.520958 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fafaf1-dd68-47d5-9059-be7ca017fa09 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:13:47.527292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080dcd8-b0cd-4d99-b3c3-af5cc6f28932 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:13:47.528512 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080dcd8-b0cd-4d99-b3c3-af5cc6f28932 service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:47.528512 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080dcd8-b0cd-4d99-b3c3-af5cc6f28932 service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:47.528748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080dcd8-b0cd-4d99-b3c3-af5cc6f28932 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:47.530959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-5080dcd8-b0cd-4d99-b3c3-af5cc6f28932 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:13:47.545718 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080dcd8-b0cd-4d99-b3c3-af5cc6f28932 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:13:47.551778 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080dcd8-b0cd-4d99-b3c3-af5cc6f28932 service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:13:47.558429 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98694fd5-98a9-4105-8b52-61891886cf13 service placement] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:13:47.561555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-98694fd5-98a9-4105-8b52-61891886cf13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:13:47.569382 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1414/2816] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:13:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:13:47.647531 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98694fd5-98a9-4105-8b52-61891886cf13 service placement] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:13:47.652061 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6f48b-0751-4607-9470-4a0ec6b8d399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:13:47.652959 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6f48b-0751-4607-9470-4a0ec6b8d399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:47.653246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6f48b-0751-4607-9470-4a0ec6b8d399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:47.653524 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6f48b-0751-4607-9470-4a0ec6b8d399 admin admin] PATH_INFO: `/v3/users/8a7e4fed38914f91beae1c094d73163f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:47.654706 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6f48b-0751-4607-9470-4a0ec6b8d399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7e4fed38914f91beae1c094d73163f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:13:47.656650 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6f48b-0751-4607-9470-4a0ec6b8d399 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:13:47.701986 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a6a6f48b-0751-4607-9470-4a0ec6b8d399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7e4fed38914f91beae1c094d73163f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:13:47.710582 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a6a6f48b-0751-4607-9470-4a0ec6b8d399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7e4fed38914f91beae1c094d73163f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:13:47.722738 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a6a6f48b-0751-4607-9470-4a0ec6b8d399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7e4fed38914f91beae1c094d73163f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:13:47.730316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a6a6f48b-0751-4607-9470-4a0ec6b8d399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7e4fed38914f91beae1c094d73163f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:13:47.731015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a6a6f48b-0751-4607-9470-4a0ec6b8d399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7e4fed38914f91beae1c094d73163f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:13:47.737068 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1403/2817] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:13:47 2022] DELETE /identity/v3/users/8a7e4fed38914f91beae1c094d73163f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:13:47.919524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080dcd8-b0cd-4d99-b3c3-af5cc6f28932 service placement] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:13:47.933990 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080dcd8-b0cd-4d99-b3c3-af5cc6f28932 service placement] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:13:47.997644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c4183-41b6-49d8-a270-1b4ad6890973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:13:47.998517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c4183-41b6-49d8-a270-1b4ad6890973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:47.998801 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c4183-41b6-49d8-a270-1b4ad6890973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:47.999076 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c4183-41b6-49d8-a270-1b4ad6890973 admin admin] PATH_INFO: `/v3/projects/37eee429bcbb4cdabc59a2e8a23da997` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:48.000261 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c4183-41b6-49d8-a270-1b4ad6890973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37eee429bcbb4cdabc59a2e8a23da997)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:13:48.002187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c4183-41b6-49d8-a270-1b4ad6890973 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:13:48.048445 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f06c4183-41b6-49d8-a270-1b4ad6890973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37eee429bcbb4cdabc59a2e8a23da997', 'request_id': 'req-f06c4183-41b6-49d8-a270-1b4ad6890973'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:13:48.055930 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f06c4183-41b6-49d8-a270-1b4ad6890973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37eee429bcbb4cdabc59a2e8a23da997', 'request_id': 'req-f06c4183-41b6-49d8-a270-1b4ad6890973'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:13:48.061756 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f06c4183-41b6-49d8-a270-1b4ad6890973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37eee429bcbb4cdabc59a2e8a23da997', 'request_id': 'req-f06c4183-41b6-49d8-a270-1b4ad6890973'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:13:48.073617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f06c4183-41b6-49d8-a270-1b4ad6890973 admin admin] The token cache is being invalidate because project 37eee429bcbb4cdabc59a2e8a23da997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:13:48.073907 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f06c4183-41b6-49d8-a270-1b4ad6890973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:13:48.074548 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-f06c4183-41b6-49d8-a270-1b4ad6890973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:13:48.076225 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1415/2818] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:13:47 2022] DELETE /identity/v3/projects/37eee429bcbb4cdabc59a2e8a23da997 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:13:48.081203 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7632bc-0f77-47c4-ab2e-b920de527f36 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:48.081516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7632bc-0f77-47c4-ab2e-b920de527f36 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:48.081797 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7632bc-0f77-47c4-ab2e-b920de527f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:48.101294 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6f7632bc-0f77-47c4-ab2e-b920de527f36 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:13:48.102431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6f7632bc-0f77-47c4-ab2e-b920de527f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:13:48.162363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1404/2819] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:13:48.170171 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439698f-954a-4d36-978a-3e008ffa64f5 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:48.170462 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439698f-954a-4d36-978a-3e008ffa64f5 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:48.170745 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439698f-954a-4d36-978a-3e008ffa64f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:48.188905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-d439698f-954a-4d36-978a-3e008ffa64f5 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:13:48.190110 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d439698f-954a-4d36-978a-3e008ffa64f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:13:48.198079 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1416/2820] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:13:48.202201 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7632bc-0f77-47c4-ab2e-b920de527f36 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:13:48.212007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7632bc-0f77-47c4-ab2e-b920de527f36 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:13:48.216603 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed377e-48bc-431b-a30e-ec9d34eae59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:13:48.217253 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed377e-48bc-431b-a30e-ec9d34eae59a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:48.217537 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed377e-48bc-431b-a30e-ec9d34eae59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:48.217814 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed377e-48bc-431b-a30e-ec9d34eae59a admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:48.218566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed377e-48bc-431b-a30e-ec9d34eae59a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:13:48.218872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed377e-48bc-431b-a30e-ec9d34eae59a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:13:48.220589 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed377e-48bc-431b-a30e-ec9d34eae59a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:13:48.232639 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1405/2821] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 23:13:48.236900 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439698f-954a-4d36-978a-3e008ffa64f5 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:13:48.247034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439698f-954a-4d36-978a-3e008ffa64f5 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:13:48.251754 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1481f3-3935-4a22-8dfe-96d75958a869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:13:48.252413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1481f3-3935-4a22-8dfe-96d75958a869 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:48.252744 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1481f3-3935-4a22-8dfe-96d75958a869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:48.253028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1481f3-3935-4a22-8dfe-96d75958a869 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:48.253734 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1481f3-3935-4a22-8dfe-96d75958a869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:13:48.255631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1481f3-3935-4a22-8dfe-96d75958a869 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:13:48.280928 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1417/2822] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:13:48 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:13:48.285089 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed377e-48bc-431b-a30e-ec9d34eae59a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:13:48.295750 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed377e-48bc-431b-a30e-ec9d34eae59a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:13:48.299557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:13:48.355882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186ced7-fce4-424c-9ff8-caf21e60a527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:13:48.356503 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186ced7-fce4-424c-9ff8-caf21e60a527 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:48.356794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186ced7-fce4-424c-9ff8-caf21e60a527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:48.357076 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186ced7-fce4-424c-9ff8-caf21e60a527 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:48.357774 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186ced7-fce4-424c-9ff8-caf21e60a527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:13:48.359705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186ced7-fce4-424c-9ff8-caf21e60a527 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:13:48.379568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1406/2823] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:13:48 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:13:48.383740 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1481f3-3935-4a22-8dfe-96d75958a869 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:13:48.394071 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1481f3-3935-4a22-8dfe-96d75958a869 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:13:48.399439 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2eba5-a7c7-4b7f-b85e-ebbcdc826e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:13:48.400133 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2eba5-a7c7-4b7f-b85e-ebbcdc826e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:48.400423 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2eba5-a7c7-4b7f-b85e-ebbcdc826e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:48.400733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2eba5-a7c7-4b7f-b85e-ebbcdc826e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:48.401477 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2eba5-a7c7-4b7f-b85e-ebbcdc826e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:13:48.403073 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2eba5-a7c7-4b7f-b85e-ebbcdc826e42 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:13:48.413813 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1418/2824] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:13:48.418102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186ced7-fce4-424c-9ff8-caf21e60a527 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:13:48.428717 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186ced7-fce4-424c-9ff8-caf21e60a527 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:13:48.434069 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38df5fd-51d2-4ee0-ac20-eb956fa437d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:13:48.434271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38df5fd-51d2-4ee0-ac20-eb956fa437d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:48.434553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38df5fd-51d2-4ee0-ac20-eb956fa437d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:48.434831 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38df5fd-51d2-4ee0-ac20-eb956fa437d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:48.435566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38df5fd-51d2-4ee0-ac20-eb956fa437d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:13:48.437263 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38df5fd-51d2-4ee0-ac20-eb956fa437d7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:13:48.447430 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1407/2825] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:13:48.451592 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2eba5-a7c7-4b7f-b85e-ebbcdc826e42 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:13:48.462151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2eba5-a7c7-4b7f-b85e-ebbcdc826e42 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:13:48.466732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4dfc70-81f4-484f-b39c-e0f8dead9bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:13:48.467608 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dfc70-81f4-484f-b39c-e0f8dead9bce admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:48.467894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dfc70-81f4-484f-b39c-e0f8dead9bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:48.468174 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dfc70-81f4-484f-b39c-e0f8dead9bce admin admin] PATH_INFO: `/v3/projects/dc66bd1d7d6b4d52b9f9bcd96f48b192/users/83b51eb318cf4267bbea75301a37d321/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:48.483115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4dfc70-81f4-484f-b39c-e0f8dead9bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc66bd1d7d6b4d52b9f9bcd96f48b192, user_id=83b51eb318cf4267bbea75301a37d321, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:13:48.485788 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4dfc70-81f4-484f-b39c-e0f8dead9bce admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:13:48.494884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:13:48.499722 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1419/2826] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:13:48 2022] PUT /identity/v3/projects/dc66bd1d7d6b4d52b9f9bcd96f48b192/users/83b51eb318cf4267bbea75301a37d321/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:13:48.505701 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916cae4-ff86-4216-ae20-6c65a9b9f39a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:48.505997 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916cae4-ff86-4216-ae20-6c65a9b9f39a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:48.506270 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916cae4-ff86-4216-ae20-6c65a9b9f39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:48.524079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6916cae4-ff86-4216-ae20-6c65a9b9f39a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:13:48.525425 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6916cae4-ff86-4216-ae20-6c65a9b9f39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:13:48.586312 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1408/2827] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:13:48.595818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4dfc70-81f4-484f-b39c-e0f8dead9bce admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:13:48.597572 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4d4dfc70-81f4-484f-b39c-e0f8dead9bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:13:48.654544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4dfc70-81f4-484f-b39c-e0f8dead9bce admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:13:48.658435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168ebee-03b4-4e24-835c-b51c818c4474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:13:48.659073 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168ebee-03b4-4e24-835c-b51c818c4474 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:48.659359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168ebee-03b4-4e24-835c-b51c818c4474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:48.659636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168ebee-03b4-4e24-835c-b51c818c4474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:48.669140 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168ebee-03b4-4e24-835c-b51c818c4474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:13:48.670567 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168ebee-03b4-4e24-835c-b51c818c4474 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:13:48.680939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1420/2828] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:13:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:13:52.612072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fbbb7-6bb3-4d33-a659-c901fa14abd0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:52.612555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fbbb7-6bb3-4d33-a659-c901fa14abd0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:52.612692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fbbb7-6bb3-4d33-a659-c901fa14abd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:52.642377 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-586fbbb7-6bb3-4d33-a659-c901fa14abd0 None None] MFA Rules not processed for user `83b51eb318cf4267bbea75301a37d321`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:13:52.643882 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-586fbbb7-6bb3-4d33-a659-c901fa14abd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:13:52.729618 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1409/2829] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:13:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:13:52.741483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168ebee-03b4-4e24-835c-b51c818c4474 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:13:52.742996 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c168ebee-03b4-4e24-835c-b51c818c4474 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:13:52.840166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168ebee-03b4-4e24-835c-b51c818c4474 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:13:52.847579 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639646c-0cba-4bb1-91c6-c90631b08494 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:13:52.847765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639646c-0cba-4bb1-91c6-c90631b08494 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:52.849025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639646c-0cba-4bb1-91c6-c90631b08494 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:52.849310 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639646c-0cba-4bb1-91c6-c90631b08494 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:52.858903 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639646c-0cba-4bb1-91c6-c90631b08494 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:13:52.860588 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639646c-0cba-4bb1-91c6-c90631b08494 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:13:52.871401 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1421/2830] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:13:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:13:54.183662 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03ccc0-6165-4f2d-9126-432296c41b8b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:13:54.184173 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03ccc0-6165-4f2d-9126-432296c41b8b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:13:54.184305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03ccc0-6165-4f2d-9126-432296c41b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:13:54.203666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7a03ccc0-6165-4f2d-9126-432296c41b8b None None] MFA Rules not processed for user `83b51eb318cf4267bbea75301a37d321`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:13:54.205045 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7a03ccc0-6165-4f2d-9126-432296c41b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:13:54.212872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1410/2831] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:13:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:14:02.866708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=67 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:14:04.198005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:14:53.065934 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b65238-5390-4f75-b340-8736522c77d8 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:14:53.066500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b65238-5390-4f75-b340-8736522c77d8 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:14:53.066500 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b65238-5390-4f75-b340-8736522c77d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:14:53.095667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-26b65238-5390-4f75-b340-8736522c77d8 None None] MFA Rules not processed for user `83b51eb318cf4267bbea75301a37d321`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:14:53.096895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-26b65238-5390-4f75-b340-8736522c77d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:14:53.107591 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1422/2832] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:14:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:14:53.414065 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03ccc0-6165-4f2d-9126-432296c41b8b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:14:53.434805 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03ccc0-6165-4f2d-9126-432296c41b8b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:14:53.438439 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c707f2f-5595-4455-b784-c6b193f90f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:14:53.439082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c707f2f-5595-4455-b784-c6b193f90f7f service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:14:53.439364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c707f2f-5595-4455-b784-c6b193f90f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:14:53.439640 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c707f2f-5595-4455-b784-c6b193f90f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:14:53.441433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4c707f2f-5595-4455-b784-c6b193f90f7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:14:53.451277 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c707f2f-5595-4455-b784-c6b193f90f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:14:53.452742 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c707f2f-5595-4455-b784-c6b193f90f7f service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:14:53.460886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1411/2833] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:14:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:14:53.783332 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b65238-5390-4f75-b340-8736522c77d8 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:14:53.784910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-26b65238-5390-4f75-b340-8736522c77d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:14:53.854879 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b65238-5390-4f75-b340-8736522c77d8 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:14:53.859859 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f08725-6d07-4172-9cb4-a63ec2a12990 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:14:53.860547 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f08725-6d07-4172-9cb4-a63ec2a12990 service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:14:53.860839 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f08725-6d07-4172-9cb4-a63ec2a12990 service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:14:53.861119 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f08725-6d07-4172-9cb4-a63ec2a12990 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:14:53.863163 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c8f08725-6d07-4172-9cb4-a63ec2a12990 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:14:53.874835 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f08725-6d07-4172-9cb4-a63ec2a12990 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:14:53.876431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f08725-6d07-4172-9cb4-a63ec2a12990 service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:14:53.886569 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1423/2834] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:14:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:14:56.491508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c707f2f-5595-4455-b784-c6b193f90f7f service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:14:56.510919 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c707f2f-5595-4455-b784-c6b193f90f7f service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:14:56.516998 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c74a78-c673-4959-9cc0-1b15e3ec54ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:14:56.517747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c74a78-c673-4959-9cc0-1b15e3ec54ba service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:14:56.518101 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c74a78-c673-4959-9cc0-1b15e3ec54ba service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:14:56.518381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c74a78-c673-4959-9cc0-1b15e3ec54ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:14:56.520307 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e0c74a78-c673-4959-9cc0-1b15e3ec54ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:14:56.535588 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c74a78-c673-4959-9cc0-1b15e3ec54ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:14:56.538010 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c74a78-c673-4959-9cc0-1b15e3ec54ba service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:14:56.559449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1412/2835] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:14:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:15:01.302623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f08725-6d07-4172-9cb4-a63ec2a12990 service placement] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:15:01.304607 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c8f08725-6d07-4172-9cb4-a63ec2a12990 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:15:01.320927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f08725-6d07-4172-9cb4-a63ec2a12990 service placement] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:15:01.327583 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e0ee1-169a-4c87-bf93-dfba2d22c116 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:15:01.328237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0ee1-169a-4c87-bf93-dfba2d22c116 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:15:01.328544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0ee1-169a-4c87-bf93-dfba2d22c116 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:15:01.328824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e0ee1-169a-4c87-bf93-dfba2d22c116 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:15:01.330568 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-750e0ee1-169a-4c87-bf93-dfba2d22c116 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:15:01.343668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e0ee1-169a-4c87-bf93-dfba2d22c116 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:15:01.345288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e0ee1-169a-4c87-bf93-dfba2d22c116 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:15:01.356029 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1424/2836] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:15:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:15:06.545071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=17 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:15:11.351557 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=47 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:15:53.786330 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860e5dc-e87e-4c2b-80d1-f0d94da517ef None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:15:53.786845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860e5dc-e87e-4c2b-80d1-f0d94da517ef None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:15:53.786957 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860e5dc-e87e-4c2b-80d1-f0d94da517ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:15:53.805158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9860e5dc-e87e-4c2b-80d1-f0d94da517ef None None] MFA Rules not processed for user `83b51eb318cf4267bbea75301a37d321`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:15:53.806384 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9860e5dc-e87e-4c2b-80d1-f0d94da517ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:15:53.813285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1413/2837] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:15:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:16:03.800225 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:16:05.045110 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e0ee1-169a-4c87-bf93-dfba2d22c116 service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:16:05.057236 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e0ee1-169a-4c87-bf93-dfba2d22c116 service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:16:05.062956 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1a560-012d-4b9e-8295-4432f471b832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:16:05.063609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1a560-012d-4b9e-8295-4432f471b832 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:16:05.063900 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1a560-012d-4b9e-8295-4432f471b832 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:16:05.064191 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1a560-012d-4b9e-8295-4432f471b832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:16:05.065997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-72a1a560-012d-4b9e-8295-4432f471b832 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:16:05.079111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1a560-012d-4b9e-8295-4432f471b832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:16:05.080716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a1a560-012d-4b9e-8295-4432f471b832 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:16:05.092008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1425/2838] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:16:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:16:15.087508 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=6 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:16:50.026202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860e5dc-e87e-4c2b-80d1-f0d94da517ef None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:16:50.028119 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9860e5dc-e87e-4c2b-80d1-f0d94da517ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:16:50.105121 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860e5dc-e87e-4c2b-80d1-f0d94da517ef None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:16:50.108916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82e67-15c6-40b4-8be7-a4aa6c473897 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:16:50.109553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82e67-15c6-40b4-8be7-a4aa6c473897 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:16:50.109836 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82e67-15c6-40b4-8be7-a4aa6c473897 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:16:50.110130 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82e67-15c6-40b4-8be7-a4aa6c473897 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:16:50.111871 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c8e82e67-15c6-40b4-8be7-a4aa6c473897 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:16:50.125316 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e82e67-15c6-40b4-8be7-a4aa6c473897 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:16:50.126785 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e82e67-15c6-40b4-8be7-a4aa6c473897 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:16:50.138076 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1414/2839] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:16:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:16:57.399416 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98eb2a-c658-45ac-8ecb-94b1fedcdd94 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:16:57.399939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98eb2a-c658-45ac-8ecb-94b1fedcdd94 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:16:57.400057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98eb2a-c658-45ac-8ecb-94b1fedcdd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:16:57.418891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-1f98eb2a-c658-45ac-8ecb-94b1fedcdd94 None None] MFA Rules not processed for user `83b51eb318cf4267bbea75301a37d321`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:16:57.420785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-1f98eb2a-c658-45ac-8ecb-94b1fedcdd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:16:57.427645 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1426/2840] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:16:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:17:00.132775 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=35 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:17:07.417499 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:17:21.207242 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82e67-15c6-40b4-8be7-a4aa6c473897 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:17:21.208736 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c8e82e67-15c6-40b4-8be7-a4aa6c473897 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:17:21.241303 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82e67-15c6-40b4-8be7-a4aa6c473897 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:17:21.244856 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb93dc-e43b-4e06-8aba-a214a9b30706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:17:21.245489 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb93dc-e43b-4e06-8aba-a214a9b30706 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:17:21.245768 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb93dc-e43b-4e06-8aba-a214a9b30706 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:17:21.246041 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb93dc-e43b-4e06-8aba-a214a9b30706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:17:21.247799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-33bb93dc-e43b-4e06-8aba-a214a9b30706 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:17:21.257631 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb93dc-e43b-4e06-8aba-a214a9b30706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:17:21.259049 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb93dc-e43b-4e06-8aba-a214a9b30706 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:17:21.267556 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1415/2841] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:17:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:17:31.263361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=15 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:17:53.919266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98eb2a-c658-45ac-8ecb-94b1fedcdd94 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:17:53.928819 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98eb2a-c658-45ac-8ecb-94b1fedcdd94 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:17:53.932330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a56e0-780c-4a37-a061-18dd1e7b9369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:17:53.933010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a56e0-780c-4a37-a061-18dd1e7b9369 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:17:53.933290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a56e0-780c-4a37-a061-18dd1e7b9369 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:17:53.933565 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a56e0-780c-4a37-a061-18dd1e7b9369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:17:53.935299 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-002a56e0-780c-4a37-a061-18dd1e7b9369 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:17:53.948625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a56e0-780c-4a37-a061-18dd1e7b9369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:17:53.949965 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a56e0-780c-4a37-a061-18dd1e7b9369 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:17:53.961229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1427/2842] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:17:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:18:03.955876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=6 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:06.203725 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb93dc-e43b-4e06-8aba-a214a9b30706 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:06.214040 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-33bb93dc-e43b-4e06-8aba-a214a9b30706 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:06.250300 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb93dc-e43b-4e06-8aba-a214a9b30706 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:06.256199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87528b8-eed8-483e-a117-b7d993998819 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:06.256903 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87528b8-eed8-483e-a117-b7d993998819 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:06.257188 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87528b8-eed8-483e-a117-b7d993998819 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:06.257460 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87528b8-eed8-483e-a117-b7d993998819 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:06.259252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a87528b8-eed8-483e-a117-b7d993998819 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:06.273054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87528b8-eed8-483e-a117-b7d993998819 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:06.275362 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87528b8-eed8-483e-a117-b7d993998819 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:06.280874 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a56e0-780c-4a37-a061-18dd1e7b9369 service cinder] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:06.289613 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1416/2843] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:18:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:18:06.324024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a56e0-780c-4a37-a061-18dd1e7b9369 service cinder] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:06.328801 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be208eb8-e45b-49ee-8bbf-96e3627e5cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:06.329712 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be208eb8-e45b-49ee-8bbf-96e3627e5cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:06.330001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be208eb8-e45b-49ee-8bbf-96e3627e5cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:06.330282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be208eb8-e45b-49ee-8bbf-96e3627e5cd5 admin admin] PATH_INFO: `/v3/users/83b51eb318cf4267bbea75301a37d321` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:06.332937 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be208eb8-e45b-49ee-8bbf-96e3627e5cd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b51eb318cf4267bbea75301a37d321)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:06.334972 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be208eb8-e45b-49ee-8bbf-96e3627e5cd5 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:06.384118 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-be208eb8-e45b-49ee-8bbf-96e3627e5cd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b51eb318cf4267bbea75301a37d321'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:06.392149 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-be208eb8-e45b-49ee-8bbf-96e3627e5cd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b51eb318cf4267bbea75301a37d321'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:06.392821 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-be208eb8-e45b-49ee-8bbf-96e3627e5cd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b51eb318cf4267bbea75301a37d321'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:06.399672 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-be208eb8-e45b-49ee-8bbf-96e3627e5cd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b51eb318cf4267bbea75301a37d321'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:06.400315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-be208eb8-e45b-49ee-8bbf-96e3627e5cd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b51eb318cf4267bbea75301a37d321'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:06.417910 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1428/2844] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:18:06 2022] DELETE /identity/v3/users/83b51eb318cf4267bbea75301a37d321 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:18:06.588054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87528b8-eed8-483e-a117-b7d993998819 service placement] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:06.599454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87528b8-eed8-483e-a117-b7d993998819 service placement] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:06.658054 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9ecf8-844d-466a-b83e-f6690061815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:06.658916 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9ecf8-844d-466a-b83e-f6690061815d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:06.659200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9ecf8-844d-466a-b83e-f6690061815d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:06.659475 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9ecf8-844d-466a-b83e-f6690061815d admin admin] PATH_INFO: `/v3/projects/dc66bd1d7d6b4d52b9f9bcd96f48b192` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:06.661367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9ecf8-844d-466a-b83e-f6690061815d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc66bd1d7d6b4d52b9f9bcd96f48b192)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:06.663258 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9ecf8-844d-466a-b83e-f6690061815d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:06.708539 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-baa9ecf8-844d-466a-b83e-f6690061815d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc66bd1d7d6b4d52b9f9bcd96f48b192', 'request_id': 'req-baa9ecf8-844d-466a-b83e-f6690061815d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:06.714077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-baa9ecf8-844d-466a-b83e-f6690061815d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc66bd1d7d6b4d52b9f9bcd96f48b192', 'request_id': 'req-baa9ecf8-844d-466a-b83e-f6690061815d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:06.721255 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-baa9ecf8-844d-466a-b83e-f6690061815d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc66bd1d7d6b4d52b9f9bcd96f48b192', 'request_id': 'req-baa9ecf8-844d-466a-b83e-f6690061815d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:06.745072 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-baa9ecf8-844d-466a-b83e-f6690061815d admin admin] The token cache is being invalidate because project dc66bd1d7d6b4d52b9f9bcd96f48b192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:18:06.745361 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-baa9ecf8-844d-466a-b83e-f6690061815d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:06.745983 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-baa9ecf8-844d-466a-b83e-f6690061815d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:06.747497 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1417/2845] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:18:06 2022] DELETE /identity/v3/projects/dc66bd1d7d6b4d52b9f9bcd96f48b192 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:06.752370 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd716a0e-82aa-462a-933e-6c473b6ef67a None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:06.752706 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd716a0e-82aa-462a-933e-6c473b6ef67a None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:06.752999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd716a0e-82aa-462a-933e-6c473b6ef67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:06.808946 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-dd716a0e-82aa-462a-933e-6c473b6ef67a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:06.810151 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-dd716a0e-82aa-462a-933e-6c473b6ef67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:06.867066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1429/2846] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:06.875810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8abbae-0fc4-4d9f-99fa-eb94ea0ec022 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:06.876105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8abbae-0fc4-4d9f-99fa-eb94ea0ec022 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:06.876387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8abbae-0fc4-4d9f-99fa-eb94ea0ec022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:06.894200 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9f8abbae-0fc4-4d9f-99fa-eb94ea0ec022 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:06.896223 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9f8abbae-0fc4-4d9f-99fa-eb94ea0ec022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:06.904071 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1418/2847] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:06.908174 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd716a0e-82aa-462a-933e-6c473b6ef67a None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:06.918320 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd716a0e-82aa-462a-933e-6c473b6ef67a None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:06.922834 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c03bc-461b-4d8b-8f41-adb9f0d93cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:06.923492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c03bc-461b-4d8b-8f41-adb9f0d93cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:06.923775 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c03bc-461b-4d8b-8f41-adb9f0d93cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:06.924051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c03bc-461b-4d8b-8f41-adb9f0d93cdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:06.924818 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c03bc-461b-4d8b-8f41-adb9f0d93cdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:18:06.925125 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c03bc-461b-4d8b-8f41-adb9f0d93cdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:06.926759 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c03bc-461b-4d8b-8f41-adb9f0d93cdf admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:06.938046 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1430/2848] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:18:06.942284 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8abbae-0fc4-4d9f-99fa-eb94ea0ec022 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:06.951245 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8abbae-0fc4-4d9f-99fa-eb94ea0ec022 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:06.955828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b488af4-02ee-483c-9836-415a257be1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:06.956468 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b488af4-02ee-483c-9836-415a257be1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:06.957292 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b488af4-02ee-483c-9836-415a257be1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:06.957616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b488af4-02ee-483c-9836-415a257be1b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:06.958309 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b488af4-02ee-483c-9836-415a257be1b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:06.960187 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b488af4-02ee-483c-9836-415a257be1b7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:06.994381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1419/2849] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:18:06 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:18:06.998544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c03bc-461b-4d8b-8f41-adb9f0d93cdf admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:07.008419 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c03bc-461b-4d8b-8f41-adb9f0d93cdf admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:07.017753 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:07.065742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67794f-d3b2-4adf-b64b-7e9b91f48101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:07.066330 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67794f-d3b2-4adf-b64b-7e9b91f48101 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:07.066617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67794f-d3b2-4adf-b64b-7e9b91f48101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:07.066893 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67794f-d3b2-4adf-b64b-7e9b91f48101 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:07.067589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67794f-d3b2-4adf-b64b-7e9b91f48101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:07.069516 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67794f-d3b2-4adf-b64b-7e9b91f48101 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:07.089115 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1431/2850] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:18:06 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:18:07.093244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b488af4-02ee-483c-9836-415a257be1b7 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:07.103978 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b488af4-02ee-483c-9836-415a257be1b7 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:07.108506 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fe5d5-cd40-4513-b0fa-94a703b9b961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:07.109172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe5d5-cd40-4513-b0fa-94a703b9b961 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:07.109454 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe5d5-cd40-4513-b0fa-94a703b9b961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:07.109729 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe5d5-cd40-4513-b0fa-94a703b9b961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:07.110452 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47fe5d5-cd40-4513-b0fa-94a703b9b961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:07.112030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47fe5d5-cd40-4513-b0fa-94a703b9b961 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:07.126182 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1420/2851] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:18:07.130708 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67794f-d3b2-4adf-b64b-7e9b91f48101 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:07.140617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67794f-d3b2-4adf-b64b-7e9b91f48101 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:07.146034 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79823505-6032-48c6-86ff-edb82e955dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:07.146690 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79823505-6032-48c6-86ff-edb82e955dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:07.146977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79823505-6032-48c6-86ff-edb82e955dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:07.147478 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79823505-6032-48c6-86ff-edb82e955dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:07.148215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79823505-6032-48c6-86ff-edb82e955dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:07.149821 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79823505-6032-48c6-86ff-edb82e955dfb admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:07.159483 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1432/2852] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:18:07.163484 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fe5d5-cd40-4513-b0fa-94a703b9b961 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:07.173280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fe5d5-cd40-4513-b0fa-94a703b9b961 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:07.178731 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161d09a-070d-470f-9bb1-054328d862db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:07.179598 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161d09a-070d-470f-9bb1-054328d862db admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:07.179886 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161d09a-070d-470f-9bb1-054328d862db admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:07.180163 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161d09a-070d-470f-9bb1-054328d862db admin admin] PATH_INFO: `/v3/projects/6fbca9d8ee92456dbdeb33f574af9415/users/b38a59b99c024d63a1be3bd9055c4cc3/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:07.182420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:07.196839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161d09a-070d-470f-9bb1-054328d862db admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fbca9d8ee92456dbdeb33f574af9415, user_id=b38a59b99c024d63a1be3bd9055c4cc3, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:07.199480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161d09a-070d-470f-9bb1-054328d862db admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:07.212102 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1421/2853] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:18:07 2022] PUT /identity/v3/projects/6fbca9d8ee92456dbdeb33f574af9415/users/b38a59b99c024d63a1be3bd9055c4cc3/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:18:07.222113 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e678-b234-4183-8c75-48f5a648dbb2 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:07.222936 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e678-b234-4183-8c75-48f5a648dbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:07.223362 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e678-b234-4183-8c75-48f5a648dbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:07.249878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-d145e678-b234-4183-8c75-48f5a648dbb2 None None] MFA Rules not processed for user `b38a59b99c024d63a1be3bd9055c4cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:07.251066 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d145e678-b234-4183-8c75-48f5a648dbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:07.314306 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1433/2854] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:07.318946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161d09a-070d-470f-9bb1-054328d862db admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:07.381495 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161d09a-070d-470f-9bb1-054328d862db admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:07.386304 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea96f98-cff3-4e87-a179-fd2981eed838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:07.386972 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea96f98-cff3-4e87-a179-fd2981eed838 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:07.387251 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea96f98-cff3-4e87-a179-fd2981eed838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:07.387553 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea96f98-cff3-4e87-a179-fd2981eed838 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:07.388249 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea96f98-cff3-4e87-a179-fd2981eed838 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:07.390226 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea96f98-cff3-4e87-a179-fd2981eed838 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:07.416403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1422/2855] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:18:07 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:18:07.421582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145e678-b234-4183-8c75-48f5a648dbb2 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:07.432675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145e678-b234-4183-8c75-48f5a648dbb2 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:07.488075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7752c5-e778-4ecd-9f37-899b899588fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:07.488922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7752c5-e778-4ecd-9f37-899b899588fd admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:07.489209 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7752c5-e778-4ecd-9f37-899b899588fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:07.489486 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7752c5-e778-4ecd-9f37-899b899588fd admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:07.490190 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7752c5-e778-4ecd-9f37-899b899588fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:07.492057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7752c5-e778-4ecd-9f37-899b899588fd admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:07.513264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1434/2856] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:18:07 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:18:07.516435 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea96f98-cff3-4e87-a179-fd2981eed838 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:07.527472 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea96f98-cff3-4e87-a179-fd2981eed838 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:07.532092 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122e641-cf85-4d30-ae6c-4ad51c079c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:07.532787 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122e641-cf85-4d30-ae6c-4ad51c079c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:07.533078 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122e641-cf85-4d30-ae6c-4ad51c079c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:07.533359 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122e641-cf85-4d30-ae6c-4ad51c079c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:07.534090 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122e641-cf85-4d30-ae6c-4ad51c079c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:07.535700 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122e641-cf85-4d30-ae6c-4ad51c079c21 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:07.668137 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1423/2857] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:18:07.672403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7752c5-e778-4ecd-9f37-899b899588fd admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:07.682997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7752c5-e778-4ecd-9f37-899b899588fd admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:07.687573 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4579599-619c-4d2b-b22b-2c25bedc733a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:07.688452 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4579599-619c-4d2b-b22b-2c25bedc733a admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:07.688770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4579599-619c-4d2b-b22b-2c25bedc733a admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:07.689043 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4579599-619c-4d2b-b22b-2c25bedc733a admin admin] PATH_INFO: `/v3/projects/c015356e61a04ee6ad65b53672a1f8ab/users/e34c1a8273e94209a62dbc7e67d56455/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:07.699664 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:07.707539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4579599-619c-4d2b-b22b-2c25bedc733a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c015356e61a04ee6ad65b53672a1f8ab, user_id=e34c1a8273e94209a62dbc7e67d56455, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:07.710219 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4579599-619c-4d2b-b22b-2c25bedc733a admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:07.723728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1435/2858] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:18:07 2022] PUT /identity/v3/projects/c015356e61a04ee6ad65b53672a1f8ab/users/e34c1a8273e94209a62dbc7e67d56455/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:07.727755 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122e641-cf85-4d30-ae6c-4ad51c079c21 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:07.737609 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122e641-cf85-4d30-ae6c-4ad51c079c21 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:07.794686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe7834-8421-48ae-aeaa-b25df1e3c953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:07.795342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe7834-8421-48ae-aeaa-b25df1e3c953 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:07.795622 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe7834-8421-48ae-aeaa-b25df1e3c953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:07.795894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe7834-8421-48ae-aeaa-b25df1e3c953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:07.796663 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe7834-8421-48ae-aeaa-b25df1e3c953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:07.798230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe7834-8421-48ae-aeaa-b25df1e3c953 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:07.808331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1424/2859] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:18:07.812457 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4579599-619c-4d2b-b22b-2c25bedc733a admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:07.823229 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4579599-619c-4d2b-b22b-2c25bedc733a admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:07.827767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8150a-9986-4428-aed3-751ba3859314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:07.828670 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8150a-9986-4428-aed3-751ba3859314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:07.828957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8150a-9986-4428-aed3-751ba3859314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:07.829231 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8150a-9986-4428-aed3-751ba3859314 admin admin] PATH_INFO: `/v3/projects/c015356e61a04ee6ad65b53672a1f8ab/users/e34c1a8273e94209a62dbc7e67d56455/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:07.831173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8150a-9986-4428-aed3-751ba3859314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c015356e61a04ee6ad65b53672a1f8ab, user_id=e34c1a8273e94209a62dbc7e67d56455, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:07.833975 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8150a-9986-4428-aed3-751ba3859314 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:07.844412 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1436/2860] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:18:07 2022] PUT /identity/v3/projects/c015356e61a04ee6ad65b53672a1f8ab/users/e34c1a8273e94209a62dbc7e67d56455/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:18:07.856244 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de77d7c-9bfb-446d-84fa-ad62c98f89f3 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:07.856564 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de77d7c-9bfb-446d-84fa-ad62c98f89f3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:07.856845 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de77d7c-9bfb-446d-84fa-ad62c98f89f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:07.885162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5de77d7c-9bfb-446d-84fa-ad62c98f89f3 None None] MFA Rules not processed for user `e34c1a8273e94209a62dbc7e67d56455`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:07.886388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5de77d7c-9bfb-446d-84fa-ad62c98f89f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:07.927388 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:07.954313 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1425/2861] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:07 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:07.965302 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8150a-9986-4428-aed3-751ba3859314 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:07.966961 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bec8150a-9986-4428-aed3-751ba3859314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:08.025813 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8150a-9986-4428-aed3-751ba3859314 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:08.029444 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec69e7f-d08d-46b3-9b17-c1d803ed2da6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:08.030087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec69e7f-d08d-46b3-9b17-c1d803ed2da6 service cinder] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:08.030403 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec69e7f-d08d-46b3-9b17-c1d803ed2da6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:08.030684 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec69e7f-d08d-46b3-9b17-c1d803ed2da6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:08.040067 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec69e7f-d08d-46b3-9b17-c1d803ed2da6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:08.041529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec69e7f-d08d-46b3-9b17-c1d803ed2da6 service cinder] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:08.051513 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1437/2862] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:18:07 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:18:09.337890 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de77d7c-9bfb-446d-84fa-ad62c98f89f3 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:09.398419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de77d7c-9bfb-446d-84fa-ad62c98f89f3 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:09.402824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:09.403675 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:09.403953 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:09.404227 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] PATH_INFO: `/v3/users/b38a59b99c024d63a1be3bd9055c4cc3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:09.405420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b38a59b99c024d63a1be3bd9055c4cc3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:09.407331 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:09.453456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b38a59b99c024d63a1be3bd9055c4cc3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:09.460289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b38a59b99c024d63a1be3bd9055c4cc3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:09.471955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b38a59b99c024d63a1be3bd9055c4cc3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:09.480214 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b38a59b99c024d63a1be3bd9055c4cc3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:09.480844 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b38a59b99c024d63a1be3bd9055c4cc3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:09.489042 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1426/2863] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:18:09 2022] DELETE /identity/v3/users/b38a59b99c024d63a1be3bd9055c4cc3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:09.493849 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071f56a-a89d-4cd2-a656-af084cd1ad8d None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:09.494360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071f56a-a89d-4cd2-a656-af084cd1ad8d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:09.494468 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071f56a-a89d-4cd2-a656-af084cd1ad8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:09.513039 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-a071f56a-a89d-4cd2-a656-af084cd1ad8d None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:09.514242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a071f56a-a89d-4cd2-a656-af084cd1ad8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:09.571884 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1438/2864] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:18:09.581692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:09.583788 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:09.639944 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4541c81-3d5c-4796-a756-68af9612af1f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:09.643559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800743e-0c8d-414c-b33f-45b509a692c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:09.644190 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800743e-0c8d-414c-b33f-45b509a692c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:09.645149 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800743e-0c8d-414c-b33f-45b509a692c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:09.645398 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800743e-0c8d-414c-b33f-45b509a692c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:09.654431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800743e-0c8d-414c-b33f-45b509a692c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:09.655861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800743e-0c8d-414c-b33f-45b509a692c5 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:09.660830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:09.666007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1427/2865] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:18:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:18:09.934406 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071f56a-a89d-4cd2-a656-af084cd1ad8d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:09.935927 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-a071f56a-a89d-4cd2-a656-af084cd1ad8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:09.948940 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071f56a-a89d-4cd2-a656-af084cd1ad8d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:09.953484 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:09.954343 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:09.954627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:09.954901 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] PATH_INFO: `/v3/projects/6fbca9d8ee92456dbdeb33f574af9415` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:09.956080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fbca9d8ee92456dbdeb33f574af9415)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:09.958369 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:10.008292 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fbca9d8ee92456dbdeb33f574af9415', 'request_id': 'req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.016957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fbca9d8ee92456dbdeb33f574af9415', 'request_id': 'req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.022939 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fbca9d8ee92456dbdeb33f574af9415', 'request_id': 'req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.030728 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:10.035200 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] The token cache is being invalidate because project 6fbca9d8ee92456dbdeb33f574af9415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:18:10.035605 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.036558 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.038717 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1439/2866] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:18:09 2022] DELETE /identity/v3/projects/6fbca9d8ee92456dbdeb33f574af9415 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:10.042819 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800743e-0c8d-414c-b33f-45b509a692c5 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:10.053114 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800743e-0c8d-414c-b33f-45b509a692c5 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:10.107947 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af6d25-4c4a-4b76-8b92-8c63596fb4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:10.108848 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af6d25-4c4a-4b76-8b92-8c63596fb4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:10.109136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af6d25-4c4a-4b76-8b92-8c63596fb4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:10.109410 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af6d25-4c4a-4b76-8b92-8c63596fb4e0 admin admin] PATH_INFO: `/v3/users/e34c1a8273e94209a62dbc7e67d56455` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:10.110585 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af6d25-4c4a-4b76-8b92-8c63596fb4e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e34c1a8273e94209a62dbc7e67d56455)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:10.112499 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af6d25-4c4a-4b76-8b92-8c63596fb4e0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:10.157780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b1af6d25-4c4a-4b76-8b92-8c63596fb4e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e34c1a8273e94209a62dbc7e67d56455'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.164718 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b1af6d25-4c4a-4b76-8b92-8c63596fb4e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e34c1a8273e94209a62dbc7e67d56455'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.176931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b1af6d25-4c4a-4b76-8b92-8c63596fb4e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e34c1a8273e94209a62dbc7e67d56455'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.184220 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b1af6d25-4c4a-4b76-8b92-8c63596fb4e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e34c1a8273e94209a62dbc7e67d56455'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.184903 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b1af6d25-4c4a-4b76-8b92-8c63596fb4e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e34c1a8273e94209a62dbc7e67d56455'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.190746 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1428/2867] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:18:10 2022] DELETE /identity/v3/users/e34c1a8273e94209a62dbc7e67d56455 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:10.460173 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:10.461632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:10.477431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea86218-4ae0-45aa-8d9a-7c5bd27dc386 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:10.535597 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:10.536418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:10.536731 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:10.537007 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7 admin admin] PATH_INFO: `/v3/projects/c015356e61a04ee6ad65b53672a1f8ab` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:10.538189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c015356e61a04ee6ad65b53672a1f8ab)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:10.540080 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:10.586095 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c015356e61a04ee6ad65b53672a1f8ab', 'request_id': 'req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.592968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c015356e61a04ee6ad65b53672a1f8ab', 'request_id': 'req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.599555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c015356e61a04ee6ad65b53672a1f8ab', 'request_id': 'req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.610300 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7 admin admin] The token cache is being invalidate because project c015356e61a04ee6ad65b53672a1f8ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:18:10.610587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.611217 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-31a90bbf-2ac9-4399-a9bb-1d33bbfc3ee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:10.612773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1440/2868] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:18:10 2022] DELETE /identity/v3/projects/c015356e61a04ee6ad65b53672a1f8ab => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:10.618340 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31a8d1-7d84-492b-98e4-5b28512631b7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:10.618844 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31a8d1-7d84-492b-98e4-5b28512631b7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:10.618975 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31a8d1-7d84-492b-98e4-5b28512631b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:10.636671 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6b31a8d1-7d84-492b-98e4-5b28512631b7 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:10.637880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6b31a8d1-7d84-492b-98e4-5b28512631b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:10.695530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1429/2869] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:18:10.703339 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e1a3a-583e-4909-a729-eec1ee23808f None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:10.703636 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e1a3a-583e-4909-a729-eec1ee23808f None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:10.703920 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e1a3a-583e-4909-a729-eec1ee23808f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:10.723873 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-d19e1a3a-583e-4909-a729-eec1ee23808f None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:10.725106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-d19e1a3a-583e-4909-a729-eec1ee23808f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:10.733127 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1441/2870] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:10.737290 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31a8d1-7d84-492b-98e4-5b28512631b7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:10.747274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31a8d1-7d84-492b-98e4-5b28512631b7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:10.751803 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d82e85-0de0-4f5b-98d4-9b72e078d7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:10.752446 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d82e85-0de0-4f5b-98d4-9b72e078d7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:10.752758 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d82e85-0de0-4f5b-98d4-9b72e078d7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:10.753033 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d82e85-0de0-4f5b-98d4-9b72e078d7ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:10.753796 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d82e85-0de0-4f5b-98d4-9b72e078d7ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:18:10.754291 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d82e85-0de0-4f5b-98d4-9b72e078d7ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:10.755679 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d82e85-0de0-4f5b-98d4-9b72e078d7ea admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:10.766930 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1430/2871] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:18:10.771795 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e1a3a-583e-4909-a729-eec1ee23808f None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:10.781037 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e1a3a-583e-4909-a729-eec1ee23808f None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:10.786290 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e35924-2158-42ea-8830-b3d52970a6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:10.786880 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e35924-2158-42ea-8830-b3d52970a6a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:10.787162 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e35924-2158-42ea-8830-b3d52970a6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:10.787438 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e35924-2158-42ea-8830-b3d52970a6a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:10.788152 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e35924-2158-42ea-8830-b3d52970a6a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:10.790068 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e35924-2158-42ea-8830-b3d52970a6a2 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:10.814111 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1442/2872] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:18:10 2022] POST /identity/v3/projects => generated 389 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:18:10.818246 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d82e85-0de0-4f5b-98d4-9b72e078d7ea admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:10.828116 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d82e85-0de0-4f5b-98d4-9b72e078d7ea admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:10.831648 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:10.887641 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131757de-2948-4c68-b548-c40a804e71d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:10.888301 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131757de-2948-4c68-b548-c40a804e71d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:10.888612 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131757de-2948-4c68-b548-c40a804e71d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:10.888889 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131757de-2948-4c68-b548-c40a804e71d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:10.889601 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131757de-2948-4c68-b548-c40a804e71d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:10.891532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131757de-2948-4c68-b548-c40a804e71d9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:10.910872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1431/2873] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:18:10 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:18:10.921386 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e35924-2158-42ea-8830-b3d52970a6a2 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:10.932793 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e35924-2158-42ea-8830-b3d52970a6a2 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:10.937413 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd151a7-6a9f-4e84-8bd8-033f4ed63813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:10.938075 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd151a7-6a9f-4e84-8bd8-033f4ed63813 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:10.938360 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd151a7-6a9f-4e84-8bd8-033f4ed63813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:10.938637 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd151a7-6a9f-4e84-8bd8-033f4ed63813 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:10.939370 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd151a7-6a9f-4e84-8bd8-033f4ed63813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:10.941083 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd151a7-6a9f-4e84-8bd8-033f4ed63813 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:10.952433 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1443/2874] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:18:10.956310 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131757de-2948-4c68-b548-c40a804e71d9 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:10.966344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131757de-2948-4c68-b548-c40a804e71d9 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:10.970830 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de1519-cebd-4535-aaf7-c8d3530c9051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:10.971480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de1519-cebd-4535-aaf7-c8d3530c9051 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:10.971766 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de1519-cebd-4535-aaf7-c8d3530c9051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:10.972039 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de1519-cebd-4535-aaf7-c8d3530c9051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:10.973351 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de1519-cebd-4535-aaf7-c8d3530c9051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:10.974935 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de1519-cebd-4535-aaf7-c8d3530c9051 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:10.985381 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1432/2875] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:18:10.989625 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd151a7-6a9f-4e84-8bd8-033f4ed63813 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:11.000751 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd151a7-6a9f-4e84-8bd8-033f4ed63813 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:11.005886 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03d7ec-e165-41b6-a5be-bec914e883c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:11.006767 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03d7ec-e165-41b6-a5be-bec914e883c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:11.007048 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03d7ec-e165-41b6-a5be-bec914e883c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:11.007323 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03d7ec-e165-41b6-a5be-bec914e883c9 admin admin] PATH_INFO: `/v3/projects/020603e9c0554e93ab617fe318dd024b/users/52d1eddbec134c5da9a29f482bd0f9c8/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:11.022328 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03d7ec-e165-41b6-a5be-bec914e883c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=020603e9c0554e93ab617fe318dd024b, user_id=52d1eddbec134c5da9a29f482bd0f9c8, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:11.024985 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03d7ec-e165-41b6-a5be-bec914e883c9 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:11.037668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1444/2876] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:18:10 2022] PUT /identity/v3/projects/020603e9c0554e93ab617fe318dd024b/users/52d1eddbec134c5da9a29f482bd0f9c8/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:11.048100 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4036a5-b44f-4f0e-970d-26c7595ff135 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:11.048394 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4036a5-b44f-4f0e-970d-26c7595ff135 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:11.048713 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4036a5-b44f-4f0e-970d-26c7595ff135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:11.074617 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-fd4036a5-b44f-4f0e-970d-26c7595ff135 None None] MFA Rules not processed for user `52d1eddbec134c5da9a29f482bd0f9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:11.075927 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fd4036a5-b44f-4f0e-970d-26c7595ff135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:11.137878 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1433/2877] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:18:11.142288 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03d7ec-e165-41b6-a5be-bec914e883c9 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:11.171742 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:11.204566 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03d7ec-e165-41b6-a5be-bec914e883c9 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:11.209773 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c145fc-ace8-4e92-97d3-ce2a3c870d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:11.210425 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c145fc-ace8-4e92-97d3-ce2a3c870d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:11.210711 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c145fc-ace8-4e92-97d3-ce2a3c870d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:11.210989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c145fc-ace8-4e92-97d3-ce2a3c870d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:11.211675 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c145fc-ace8-4e92-97d3-ce2a3c870d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:11.213585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c145fc-ace8-4e92-97d3-ce2a3c870d67 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:11.237422 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1445/2878] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:18:11 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:18:11.241422 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4036a5-b44f-4f0e-970d-26c7595ff135 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:11.252170 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4036a5-b44f-4f0e-970d-26c7595ff135 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:11.306727 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2092688-b2cb-4da2-8d7e-d3e0289bcc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:11.307326 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2092688-b2cb-4da2-8d7e-d3e0289bcc4d admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:11.307610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2092688-b2cb-4da2-8d7e-d3e0289bcc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:11.307911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2092688-b2cb-4da2-8d7e-d3e0289bcc4d admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:11.308637 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2092688-b2cb-4da2-8d7e-d3e0289bcc4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:11.310548 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2092688-b2cb-4da2-8d7e-d3e0289bcc4d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:11.331802 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1434/2879] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:18:11 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:18:11.336582 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c145fc-ace8-4e92-97d3-ce2a3c870d67 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:11.346964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c145fc-ace8-4e92-97d3-ce2a3c870d67 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:11.351604 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482118d4-3485-4cd4-8aed-278b0423b0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:11.352264 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482118d4-3485-4cd4-8aed-278b0423b0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:11.352580 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482118d4-3485-4cd4-8aed-278b0423b0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:11.352861 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482118d4-3485-4cd4-8aed-278b0423b0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:11.353585 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482118d4-3485-4cd4-8aed-278b0423b0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:11.355177 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482118d4-3485-4cd4-8aed-278b0423b0d1 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:11.365945 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1446/2880] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:18:11.370382 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2092688-b2cb-4da2-8d7e-d3e0289bcc4d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:11.376953 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:11.381538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2092688-b2cb-4da2-8d7e-d3e0289bcc4d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:11.387411 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ddb9b-1e51-44e8-b3d7-c0f4891dddb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:11.388289 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ddb9b-1e51-44e8-b3d7-c0f4891dddb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:11.388600 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ddb9b-1e51-44e8-b3d7-c0f4891dddb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:11.388874 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ddb9b-1e51-44e8-b3d7-c0f4891dddb9 admin admin] PATH_INFO: `/v3/projects/1827042713e24e8081cc940804cb0ed5/users/dee445bdd45042a2a3ebfe3d08830457/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:11.403455 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ddb9b-1e51-44e8-b3d7-c0f4891dddb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1827042713e24e8081cc940804cb0ed5, user_id=dee445bdd45042a2a3ebfe3d08830457, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:11.406138 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ddb9b-1e51-44e8-b3d7-c0f4891dddb9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:11.418873 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1435/2881] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:18:11 2022] PUT /identity/v3/projects/1827042713e24e8081cc940804cb0ed5/users/dee445bdd45042a2a3ebfe3d08830457/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:11.422977 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482118d4-3485-4cd4-8aed-278b0423b0d1 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:11.433968 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482118d4-3485-4cd4-8aed-278b0423b0d1 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:11.490615 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804a2e8-08df-4b4b-b417-d41de794873f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:11.491359 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804a2e8-08df-4b4b-b417-d41de794873f admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:11.491657 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804a2e8-08df-4b4b-b417-d41de794873f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:11.491981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804a2e8-08df-4b4b-b417-d41de794873f admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:11.492755 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804a2e8-08df-4b4b-b417-d41de794873f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:11.494634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804a2e8-08df-4b4b-b417-d41de794873f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:11.505301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1447/2882] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:18:11.509396 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ddb9b-1e51-44e8-b3d7-c0f4891dddb9 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:11.520810 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ddb9b-1e51-44e8-b3d7-c0f4891dddb9 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:11.525419 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f10dc4-2782-43fb-b04d-a9d67de0ea23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:11.526301 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f10dc4-2782-43fb-b04d-a9d67de0ea23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:11.526584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f10dc4-2782-43fb-b04d-a9d67de0ea23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:11.526866 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f10dc4-2782-43fb-b04d-a9d67de0ea23 admin admin] PATH_INFO: `/v3/projects/1827042713e24e8081cc940804cb0ed5/users/dee445bdd45042a2a3ebfe3d08830457/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:11.528915 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f10dc4-2782-43fb-b04d-a9d67de0ea23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1827042713e24e8081cc940804cb0ed5, user_id=dee445bdd45042a2a3ebfe3d08830457, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:11.531549 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f10dc4-2782-43fb-b04d-a9d67de0ea23 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:11.541884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1436/2883] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:18:11 2022] PUT /identity/v3/projects/1827042713e24e8081cc940804cb0ed5/users/dee445bdd45042a2a3ebfe3d08830457/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:11.553676 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf938e7c-c3de-458d-9db5-9fcb0473259c None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:11.554166 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf938e7c-c3de-458d-9db5-9fcb0473259c None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:11.554280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf938e7c-c3de-458d-9db5-9fcb0473259c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:11.580860 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-bf938e7c-c3de-458d-9db5-9fcb0473259c None None] MFA Rules not processed for user `dee445bdd45042a2a3ebfe3d08830457`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:11.582032 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bf938e7c-c3de-458d-9db5-9fcb0473259c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:11.648392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1448/2884] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:11.658996 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f10dc4-2782-43fb-b04d-a9d67de0ea23 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:11.660872 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-61f10dc4-2782-43fb-b04d-a9d67de0ea23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:11.717623 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f10dc4-2782-43fb-b04d-a9d67de0ea23 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:11.721271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41931a0-98a5-4315-bdc2-c05e06aa3792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:11.721920 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41931a0-98a5-4315-bdc2-c05e06aa3792 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:11.722204 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41931a0-98a5-4315-bdc2-c05e06aa3792 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:11.722974 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41931a0-98a5-4315-bdc2-c05e06aa3792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:11.731918 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41931a0-98a5-4315-bdc2-c05e06aa3792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:11.733387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41931a0-98a5-4315-bdc2-c05e06aa3792 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:11.743765 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1437/2885] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:18:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:18:11.872247 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf938e7c-c3de-458d-9db5-9fcb0473259c None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:11.896682 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:11.934668 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf938e7c-c3de-458d-9db5-9fcb0473259c None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:11.939176 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:11.940051 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:11.940335 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:11.940647 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] PATH_INFO: `/v3/users/52d1eddbec134c5da9a29f482bd0f9c8` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:11.942098 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d1eddbec134c5da9a29f482bd0f9c8)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:11.944010 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:11.989545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d1eddbec134c5da9a29f482bd0f9c8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:11.996916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d1eddbec134c5da9a29f482bd0f9c8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:11.997641 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d1eddbec134c5da9a29f482bd0f9c8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.004598 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d1eddbec134c5da9a29f482bd0f9c8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.005237 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d1eddbec134c5da9a29f482bd0f9c8'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.022266 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1449/2886] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:18:11 2022] DELETE /identity/v3/users/52d1eddbec134c5da9a29f482bd0f9c8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:12.026911 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6ad3f-ac92-41f6-9a36-0df42f18d72a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:12.027202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6ad3f-ac92-41f6-9a36-0df42f18d72a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:12.027483 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6ad3f-ac92-41f6-9a36-0df42f18d72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:12.044507 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-14e6ad3f-ac92-41f6-9a36-0df42f18d72a None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:12.045719 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-14e6ad3f-ac92-41f6-9a36-0df42f18d72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:12.104557 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1438/2887] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:12.114318 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:12.116374 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:12.174346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62459b69-8aa1-4ac2-9ada-5dba57ae2f64 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:12.178008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa993cb-9b4e-44a6-8842-6cf707325520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:12.178655 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa993cb-9b4e-44a6-8842-6cf707325520 service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:12.178942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa993cb-9b4e-44a6-8842-6cf707325520 service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:12.179220 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa993cb-9b4e-44a6-8842-6cf707325520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:12.188165 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa993cb-9b4e-44a6-8842-6cf707325520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:12.189617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa993cb-9b4e-44a6-8842-6cf707325520 service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:12.199046 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1450/2888] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:18:12.477308 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6ad3f-ac92-41f6-9a36-0df42f18d72a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:12.478940 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-14e6ad3f-ac92-41f6-9a36-0df42f18d72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:12.496510 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6ad3f-ac92-41f6-9a36-0df42f18d72a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:12.501403 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:12.502399 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:12.502763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:12.502964 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] PATH_INFO: `/v3/projects/020603e9c0554e93ab617fe318dd024b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:12.504856 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=020603e9c0554e93ab617fe318dd024b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:12.506775 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:12.553311 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '020603e9c0554e93ab617fe318dd024b', 'request_id': 'req-74cf1886-bf62-40f0-8d9f-57f24fef9f97'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.558895 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '020603e9c0554e93ab617fe318dd024b', 'request_id': 'req-74cf1886-bf62-40f0-8d9f-57f24fef9f97'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.565825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '020603e9c0554e93ab617fe318dd024b', 'request_id': 'req-74cf1886-bf62-40f0-8d9f-57f24fef9f97'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.567077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:12.579965 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] The token cache is being invalidate because project 020603e9c0554e93ab617fe318dd024b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:18:12.580251 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.580931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.582828 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1439/2889] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:18:12 2022] DELETE /identity/v3/projects/020603e9c0554e93ab617fe318dd024b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:12.586923 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa993cb-9b4e-44a6-8842-6cf707325520 service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:12.596606 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa993cb-9b4e-44a6-8842-6cf707325520 service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:12.639272 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:12.655384 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa12222-33eb-4bd0-ab6e-859246c5be4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:12.656256 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa12222-33eb-4bd0-ab6e-859246c5be4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:12.656575 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa12222-33eb-4bd0-ab6e-859246c5be4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:12.656853 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa12222-33eb-4bd0-ab6e-859246c5be4a admin admin] PATH_INFO: `/v3/users/dee445bdd45042a2a3ebfe3d08830457` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:12.658024 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa12222-33eb-4bd0-ab6e-859246c5be4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dee445bdd45042a2a3ebfe3d08830457)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:12.659922 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa12222-33eb-4bd0-ab6e-859246c5be4a admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:12.706373 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-faa12222-33eb-4bd0-ab6e-859246c5be4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dee445bdd45042a2a3ebfe3d08830457'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.714994 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-faa12222-33eb-4bd0-ab6e-859246c5be4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dee445bdd45042a2a3ebfe3d08830457'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.715627 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-faa12222-33eb-4bd0-ab6e-859246c5be4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dee445bdd45042a2a3ebfe3d08830457'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.722357 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-faa12222-33eb-4bd0-ab6e-859246c5be4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dee445bdd45042a2a3ebfe3d08830457'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.723077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-faa12222-33eb-4bd0-ab6e-859246c5be4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dee445bdd45042a2a3ebfe3d08830457'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:12.739962 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1451/2890] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:18:12 2022] DELETE /identity/v3/users/dee445bdd45042a2a3ebfe3d08830457 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:13.012160 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:13.013770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:13.027540 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf1886-bf62-40f0-8d9f-57f24fef9f97 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:13.085692 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f902a71-206a-4043-b6eb-bf0810b0be5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:13.086555 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f902a71-206a-4043-b6eb-bf0810b0be5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:13.086833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f902a71-206a-4043-b6eb-bf0810b0be5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:13.087104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f902a71-206a-4043-b6eb-bf0810b0be5a admin admin] PATH_INFO: `/v3/projects/1827042713e24e8081cc940804cb0ed5` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:13.088271 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f902a71-206a-4043-b6eb-bf0810b0be5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1827042713e24e8081cc940804cb0ed5)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:13.090257 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f902a71-206a-4043-b6eb-bf0810b0be5a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:13.136467 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4f902a71-206a-4043-b6eb-bf0810b0be5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1827042713e24e8081cc940804cb0ed5', 'request_id': 'req-4f902a71-206a-4043-b6eb-bf0810b0be5a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:13.141880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4f902a71-206a-4043-b6eb-bf0810b0be5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1827042713e24e8081cc940804cb0ed5', 'request_id': 'req-4f902a71-206a-4043-b6eb-bf0810b0be5a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:13.149409 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4f902a71-206a-4043-b6eb-bf0810b0be5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1827042713e24e8081cc940804cb0ed5', 'request_id': 'req-4f902a71-206a-4043-b6eb-bf0810b0be5a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:13.160837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4f902a71-206a-4043-b6eb-bf0810b0be5a admin admin] The token cache is being invalidate because project 1827042713e24e8081cc940804cb0ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:18:13.161129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4f902a71-206a-4043-b6eb-bf0810b0be5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:13.161799 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-4f902a71-206a-4043-b6eb-bf0810b0be5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:13.163344 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1440/2891] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:18:13 2022] DELETE /identity/v3/projects/1827042713e24e8081cc940804cb0ed5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:13.169187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf60dff-988c-4e0e-a481-78b73d2aaa6b None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:13.169486 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf60dff-988c-4e0e-a481-78b73d2aaa6b None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:13.169770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf60dff-988c-4e0e-a481-78b73d2aaa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:13.188286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-ebf60dff-988c-4e0e-a481-78b73d2aaa6b None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:13.189531 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ebf60dff-988c-4e0e-a481-78b73d2aaa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:13.248077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1452/2892] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:13.255839 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd663f-7440-4681-897d-a27f0e911246 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:13.256129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd663f-7440-4681-897d-a27f0e911246 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:13.256408 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd663f-7440-4681-897d-a27f0e911246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:13.274449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-81fd663f-7440-4681-897d-a27f0e911246 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:13.275705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-81fd663f-7440-4681-897d-a27f0e911246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:13.283561 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1441/2893] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:13.288129 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf60dff-988c-4e0e-a481-78b73d2aaa6b None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:13.297643 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf60dff-988c-4e0e-a481-78b73d2aaa6b None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:13.302790 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc7295-48b5-4cb1-b28c-513b28334193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:13.303465 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc7295-48b5-4cb1-b28c-513b28334193 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:13.303750 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc7295-48b5-4cb1-b28c-513b28334193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:13.304028 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc7295-48b5-4cb1-b28c-513b28334193 admin admin] PATH_INFO: `/v3/domains` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:13.304800 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc7295-48b5-4cb1-b28c-513b28334193 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:18:13.305106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc7295-48b5-4cb1-b28c-513b28334193 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:13.306677 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc7295-48b5-4cb1-b28c-513b28334193 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:13.318153 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1453/2894] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:18:13.322180 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd663f-7440-4681-897d-a27f0e911246 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:13.331257 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd663f-7440-4681-897d-a27f0e911246 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:13.335902 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebd441-7380-47cc-b767-5a82a4efe1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:13.336579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebd441-7380-47cc-b767-5a82a4efe1ca admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:13.336865 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebd441-7380-47cc-b767-5a82a4efe1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:13.337141 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebd441-7380-47cc-b767-5a82a4efe1ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:13.337821 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ebd441-7380-47cc-b767-5a82a4efe1ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:13.339710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ebd441-7380-47cc-b767-5a82a4efe1ca admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:13.365325 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1442/2895] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:18:13 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:18:13.369571 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc7295-48b5-4cb1-b28c-513b28334193 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:13.379565 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc7295-48b5-4cb1-b28c-513b28334193 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:13.425786 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:13.437537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd5a9c-79bd-45b6-818a-028fb69537f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:13.438210 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd5a9c-79bd-45b6-818a-028fb69537f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:13.438544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd5a9c-79bd-45b6-818a-028fb69537f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:13.438824 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd5a9c-79bd-45b6-818a-028fb69537f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:13.439529 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd5a9c-79bd-45b6-818a-028fb69537f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:13.441993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd5a9c-79bd-45b6-818a-028fb69537f1 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:13.463854 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1454/2896] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:18:13 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:18:13.465532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebd441-7380-47cc-b767-5a82a4efe1ca admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:13.475825 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebd441-7380-47cc-b767-5a82a4efe1ca admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:13.480686 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b88b43-3dd8-4e27-a492-5b4c7f6e2fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:13.481342 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b88b43-3dd8-4e27-a492-5b4c7f6e2fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:13.481627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b88b43-3dd8-4e27-a492-5b4c7f6e2fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:13.481906 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b88b43-3dd8-4e27-a492-5b4c7f6e2fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:13.482634 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b88b43-3dd8-4e27-a492-5b4c7f6e2fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:13.484230 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b88b43-3dd8-4e27-a492-5b4c7f6e2fc0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:13.495593 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1443/2897] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:18:13.499336 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd5a9c-79bd-45b6-818a-028fb69537f1 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:13.509786 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd5a9c-79bd-45b6-818a-028fb69537f1 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:13.514368 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd95c61-2e80-4d21-9685-79b3922022cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:13.515018 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd95c61-2e80-4d21-9685-79b3922022cc admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:13.515301 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd95c61-2e80-4d21-9685-79b3922022cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:13.515577 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd95c61-2e80-4d21-9685-79b3922022cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:13.516295 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd95c61-2e80-4d21-9685-79b3922022cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:13.517916 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd95c61-2e80-4d21-9685-79b3922022cc admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:13.529077 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1455/2898] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:18:13.533172 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b88b43-3dd8-4e27-a492-5b4c7f6e2fc0 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:13.543274 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b88b43-3dd8-4e27-a492-5b4c7f6e2fc0 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:13.547931 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1eca3-819e-4592-97b3-2ba452aba448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:13.548824 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1eca3-819e-4592-97b3-2ba452aba448 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:13.549110 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1eca3-819e-4592-97b3-2ba452aba448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:13.549387 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1eca3-819e-4592-97b3-2ba452aba448 admin admin] PATH_INFO: `/v3/projects/143ed5c483284971abf65a42baf28bcc/users/5138b0f921774ef09dd37a3dbaddf9e0/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:13.563690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1eca3-819e-4592-97b3-2ba452aba448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=143ed5c483284971abf65a42baf28bcc, user_id=5138b0f921774ef09dd37a3dbaddf9e0, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:13.566928 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1eca3-819e-4592-97b3-2ba452aba448 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:13.579884 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1444/2899] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:18:13 2022] PUT /identity/v3/projects/143ed5c483284971abf65a42baf28bcc/users/5138b0f921774ef09dd37a3dbaddf9e0/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:18:13.591326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202da15-157f-4fa1-a906-1faae84e968d None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:13.591822 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202da15-157f-4fa1-a906-1faae84e968d None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:13.591950 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202da15-157f-4fa1-a906-1faae84e968d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:13.621633 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-b202da15-157f-4fa1-a906-1faae84e968d None None] MFA Rules not processed for user `5138b0f921774ef09dd37a3dbaddf9e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:13.622840 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b202da15-157f-4fa1-a906-1faae84e968d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:13.812816 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1456/2900] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:13.824356 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1eca3-819e-4592-97b3-2ba452aba448 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:13.826186 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b8b1eca3-819e-4592-97b3-2ba452aba448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:13.847584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:13.885483 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1eca3-819e-4592-97b3-2ba452aba448 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:13.889094 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e232609-a977-436c-bc0f-c6cad73d545e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a697d1e7e42048a8a1fb997bcc7b3fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:13.889733 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e232609-a977-436c-bc0f-c6cad73d545e service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:13.890018 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e232609-a977-436c-bc0f-c6cad73d545e service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:13.890296 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e232609-a977-436c-bc0f-c6cad73d545e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:13.898666 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e232609-a977-436c-bc0f-c6cad73d545e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:13.900097 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e232609-a977-436c-bc0f-c6cad73d545e service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:13.908794 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1445/2901] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:18:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:18:23.809895 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=43,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:23.904857 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=17 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:26.553322 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202da15-157f-4fa1-a906-1faae84e968d None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:26.554871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-b202da15-157f-4fa1-a906-1faae84e968d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:26.635192 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202da15-157f-4fa1-a906-1faae84e968d None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:26.640429 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03415d45-4407-4b87-a06c-7469ca3c5d52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:26.641132 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03415d45-4407-4b87-a06c-7469ca3c5d52 service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:26.641420 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03415d45-4407-4b87-a06c-7469ca3c5d52 service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:26.641700 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03415d45-4407-4b87-a06c-7469ca3c5d52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:26.643541 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-03415d45-4407-4b87-a06c-7469ca3c5d52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:26.656130 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03415d45-4407-4b87-a06c-7469ca3c5d52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:26.657763 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03415d45-4407-4b87-a06c-7469ca3c5d52 service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:26.668713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1457/2902] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:18:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:18:34.643933 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e232609-a977-436c-bc0f-c6cad73d545e service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:34.706421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e232609-a977-436c-bc0f-c6cad73d545e service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:34.711559 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:34.712421 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:34.712730 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:34.713005 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] PATH_INFO: `/v3/users/5138b0f921774ef09dd37a3dbaddf9e0` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:34.714208 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5138b0f921774ef09dd37a3dbaddf9e0)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:34.716131 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:34.760734 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5138b0f921774ef09dd37a3dbaddf9e0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:34.767475 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5138b0f921774ef09dd37a3dbaddf9e0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:34.779723 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5138b0f921774ef09dd37a3dbaddf9e0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:34.786989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5138b0f921774ef09dd37a3dbaddf9e0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:34.788751 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5138b0f921774ef09dd37a3dbaddf9e0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:34.794747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1446/2903] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:18:34 2022] DELETE /identity/v3/users/5138b0f921774ef09dd37a3dbaddf9e0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:34.799214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eeeb3-cfbd-4cee-a9cb-8330682720c2 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:34.799678 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eeeb3-cfbd-4cee-a9cb-8330682720c2 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:34.799799 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eeeb3-cfbd-4cee-a9cb-8330682720c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:34.817623 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-6b4eeeb3-cfbd-4cee-a9cb-8330682720c2 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:34.818823 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6b4eeeb3-cfbd-4cee-a9cb-8330682720c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:34.877785 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1458/2904] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:34.887852 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:34.889279 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:34.945560 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a89f3f-106d-4a09-9511-872724d4494d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:34.949199 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb16fc0-3a72-4d3c-8c6c-2ac13f1b9af5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:34.949837 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb16fc0-3a72-4d3c-8c6c-2ac13f1b9af5 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:34.950118 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb16fc0-3a72-4d3c-8c6c-2ac13f1b9af5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:34.950392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb16fc0-3a72-4d3c-8c6c-2ac13f1b9af5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:34.959991 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb16fc0-3a72-4d3c-8c6c-2ac13f1b9af5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:34.961534 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb16fc0-3a72-4d3c-8c6c-2ac13f1b9af5 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:34.970735 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1447/2905] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:18:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:18:35.236333 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4eeeb3-cfbd-4cee-a9cb-8330682720c2 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:35.237819 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-6b4eeeb3-cfbd-4cee-a9cb-8330682720c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:35.250099 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4eeeb3-cfbd-4cee-a9cb-8330682720c2 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:35.254355 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57602a68-a57d-4a12-9af4-43c3345fe73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:35.255216 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57602a68-a57d-4a12-9af4-43c3345fe73e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:35.255497 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57602a68-a57d-4a12-9af4-43c3345fe73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:35.255770 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57602a68-a57d-4a12-9af4-43c3345fe73e admin admin] PATH_INFO: `/v3/projects/143ed5c483284971abf65a42baf28bcc` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:35.257559 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57602a68-a57d-4a12-9af4-43c3345fe73e admin admin] RBAC: Authorizing `identity:delete_project(project_id=143ed5c483284971abf65a42baf28bcc)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:35.259476 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57602a68-a57d-4a12-9af4-43c3345fe73e admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:35.305919 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-57602a68-a57d-4a12-9af4-43c3345fe73e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '143ed5c483284971abf65a42baf28bcc', 'request_id': 'req-57602a68-a57d-4a12-9af4-43c3345fe73e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:35.312700 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-57602a68-a57d-4a12-9af4-43c3345fe73e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '143ed5c483284971abf65a42baf28bcc', 'request_id': 'req-57602a68-a57d-4a12-9af4-43c3345fe73e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:35.318545 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-57602a68-a57d-4a12-9af4-43c3345fe73e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '143ed5c483284971abf65a42baf28bcc', 'request_id': 'req-57602a68-a57d-4a12-9af4-43c3345fe73e'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:35.330214 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-57602a68-a57d-4a12-9af4-43c3345fe73e admin admin] The token cache is being invalidate because project 143ed5c483284971abf65a42baf28bcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:18:35.330510 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-57602a68-a57d-4a12-9af4-43c3345fe73e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:35.331189 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-57602a68-a57d-4a12-9af4-43c3345fe73e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:18:35.332765 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1459/2906] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:18:35 2022] DELETE /identity/v3/projects/143ed5c483284971abf65a42baf28bcc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:35.338950 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18748ac6-82f3-41cd-9aaf-f687233f4ef2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:35.339247 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18748ac6-82f3-41cd-9aaf-f687233f4ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:35.339532 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18748ac6-82f3-41cd-9aaf-f687233f4ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:35.356817 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-18748ac6-82f3-41cd-9aaf-f687233f4ef2 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:35.358063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-18748ac6-82f3-41cd-9aaf-f687233f4ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:35.375568 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:35.420610 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1448/2907] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:35.428991 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83689e6-c823-4baa-a15f-82f53a050ec3 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:35.429278 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83689e6-c823-4baa-a15f-82f53a050ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:35.429555 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83689e6-c823-4baa-a15f-82f53a050ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:35.448372 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-c83689e6-c823-4baa-a15f-82f53a050ec3 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:35.449609 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-c83689e6-c823-4baa-a15f-82f53a050ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:35.457418 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1460/2908] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:35.461946 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18748ac6-82f3-41cd-9aaf-f687233f4ef2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:35.471625 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18748ac6-82f3-41cd-9aaf-f687233f4ef2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:35.476207 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf7140-56fb-4b70-8eaf-f5da30da6312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:35.476899 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf7140-56fb-4b70-8eaf-f5da30da6312 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:35.477186 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf7140-56fb-4b70-8eaf-f5da30da6312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:35.477462 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf7140-56fb-4b70-8eaf-f5da30da6312 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:35.478196 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf7140-56fb-4b70-8eaf-f5da30da6312 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:18:35.478526 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf7140-56fb-4b70-8eaf-f5da30da6312 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:35.480152 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf7140-56fb-4b70-8eaf-f5da30da6312 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:35.492077 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1449/2909] 10.222.0.42 () {58 vars in 1152 bytes} [Thu Jul 21 23:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:18:35.496596 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83689e6-c823-4baa-a15f-82f53a050ec3 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:35.505637 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83689e6-c823-4baa-a15f-82f53a050ec3 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:35.510805 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27042f41-bc9c-435d-9cb3-1f0d3075f90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:35.511449 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27042f41-bc9c-435d-9cb3-1f0d3075f90d admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:35.511733 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27042f41-bc9c-435d-9cb3-1f0d3075f90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:35.512011 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27042f41-bc9c-435d-9cb3-1f0d3075f90d admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:35.512732 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27042f41-bc9c-435d-9cb3-1f0d3075f90d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:35.514631 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27042f41-bc9c-435d-9cb3-1f0d3075f90d admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:35.517831 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:35.540942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1461/2910] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:18:35 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:18:35.545159 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf7140-56fb-4b70-8eaf-f5da30da6312 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:35.555627 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf7140-56fb-4b70-8eaf-f5da30da6312 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:35.617780 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842c531-ba6b-4f64-99be-8e8d1c60f64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:35.618449 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842c531-ba6b-4f64-99be-8e8d1c60f64f admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:35.618732 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842c531-ba6b-4f64-99be-8e8d1c60f64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:35.619007 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842c531-ba6b-4f64-99be-8e8d1c60f64f admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:35.619948 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842c531-ba6b-4f64-99be-8e8d1c60f64f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:35.621888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842c531-ba6b-4f64-99be-8e8d1c60f64f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:35.641616 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1450/2911] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:18:35 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:18:35.645871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27042f41-bc9c-435d-9cb3-1f0d3075f90d admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:35.655980 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27042f41-bc9c-435d-9cb3-1f0d3075f90d admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:35.660466 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090ea37-c556-4e1d-8254-630a157a11bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:35.661147 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090ea37-c556-4e1d-8254-630a157a11bc admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:35.661431 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090ea37-c556-4e1d-8254-630a157a11bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:35.661707 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090ea37-c556-4e1d-8254-630a157a11bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:35.662435 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090ea37-c556-4e1d-8254-630a157a11bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:35.664666 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090ea37-c556-4e1d-8254-630a157a11bc admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:35.674492 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1462/2912] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:18:35.678984 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842c531-ba6b-4f64-99be-8e8d1c60f64f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:35.688833 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842c531-ba6b-4f64-99be-8e8d1c60f64f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:35.693431 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be6fa1-fb2f-4a7a-9858-94b5b6ddb789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:35.694082 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be6fa1-fb2f-4a7a-9858-94b5b6ddb789 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:35.694367 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be6fa1-fb2f-4a7a-9858-94b5b6ddb789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:35.694644 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be6fa1-fb2f-4a7a-9858-94b5b6ddb789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:35.695370 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be6fa1-fb2f-4a7a-9858-94b5b6ddb789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:35.696999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be6fa1-fb2f-4a7a-9858-94b5b6ddb789 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:35.706712 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1451/2913] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:18:35.710905 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090ea37-c556-4e1d-8254-630a157a11bc admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:35.720743 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090ea37-c556-4e1d-8254-630a157a11bc admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:35.725781 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eb743-56ce-4753-a6f0-6bd66f1f079f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:35.726644 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb743-56ce-4753-a6f0-6bd66f1f079f admin admin] REQUEST_METHOD: `PUT` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:35.726924 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb743-56ce-4753-a6f0-6bd66f1f079f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:35.727198 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb743-56ce-4753-a6f0-6bd66f1f079f admin admin] PATH_INFO: `/v3/projects/d72850690b7f447fbbb9a30e1bc1fc78/users/4c2cdf754ca24afd88329cd456f1c9d2/roles/1b870be20b0c4159bfbbe73593799250` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:35.741851 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eb743-56ce-4753-a6f0-6bd66f1f079f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72850690b7f447fbbb9a30e1bc1fc78, user_id=4c2cdf754ca24afd88329cd456f1c9d2, role_id=1b870be20b0c4159bfbbe73593799250)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:35.744667 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eb743-56ce-4753-a6f0-6bd66f1f079f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:35.756891 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1463/2914] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:18:35 2022] PUT /identity/v3/projects/d72850690b7f447fbbb9a30e1bc1fc78/users/4c2cdf754ca24afd88329cd456f1c9d2/roles/1b870be20b0c4159bfbbe73593799250 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:18:35.761832 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b432f29-c10b-40f4-af23-d9b2c03c89b8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:35.762176 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b432f29-c10b-40f4-af23-d9b2c03c89b8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:35.762456 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b432f29-c10b-40f4-af23-d9b2c03c89b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:35.780136 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4b432f29-c10b-40f4-af23-d9b2c03c89b8 None None] MFA Rules not processed for user `e2841614037940fea030f1d6c1eca499`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:35.781369 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4b432f29-c10b-40f4-af23-d9b2c03c89b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:35.839158 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1452/2915] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:18:35.848643 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eb743-56ce-4753-a6f0-6bd66f1f079f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:35.850981 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-ca4eb743-56ce-4753-a6f0-6bd66f1f079f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:35.906723 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eb743-56ce-4753-a6f0-6bd66f1f079f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:35.910326 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87084429-faa2-4c31-84ca-bf1d0a07c20e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:35.910964 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87084429-faa2-4c31-84ca-bf1d0a07c20e service neutron] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:35.911246 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87084429-faa2-4c31-84ca-bf1d0a07c20e service neutron] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:35.911517 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87084429-faa2-4c31-84ca-bf1d0a07c20e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:35.920989 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87084429-faa2-4c31-84ca-bf1d0a07c20e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:35.922388 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87084429-faa2-4c31-84ca-bf1d0a07c20e service neutron] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:35.932227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1464/2916] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:18:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:18:39.893363 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d34e0-6bb8-4f28-b5df-968d1a651197 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:39.893832 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d34e0-6bb8-4f28-b5df-968d1a651197 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:39.894212 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d34e0-6bb8-4f28-b5df-968d1a651197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:39.929566 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-706d34e0-6bb8-4f28-b5df-968d1a651197 None None] MFA Rules not processed for user `4c2cdf754ca24afd88329cd456f1c9d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:39.930888 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-706d34e0-6bb8-4f28-b5df-968d1a651197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:39.950530 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:40.004406 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1453/2917] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:40.005993 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87084429-faa2-4c31-84ca-bf1d0a07c20e service neutron] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:40.020016 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87084429-faa2-4c31-84ca-bf1d0a07c20e service neutron] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:40.026551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e084f-443c-476e-b219-c3b793edd423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:40.026551 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e084f-443c-476e-b219-c3b793edd423 admin admin] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:40.026805 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e084f-443c-476e-b219-c3b793edd423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:40.027057 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e084f-443c-476e-b219-c3b793edd423 admin admin] PATH_INFO: `/v3/projects` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:40.027766 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e084f-443c-476e-b219-c3b793edd423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:40.029784 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e084f-443c-476e-b219-c3b793edd423 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:40.060837 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1465/2918] 10.222.0.42 () {60 vars in 1152 bytes} [Thu Jul 21 23:18:40 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:18:40.064880 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d34e0-6bb8-4f28-b5df-968d1a651197 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:40.076463 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d34e0-6bb8-4f28-b5df-968d1a651197 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:40.151534 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e653d12-d993-44a8-b894-2c6503da6a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:40.152705 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e653d12-d993-44a8-b894-2c6503da6a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:40.153288 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e653d12-d993-44a8-b894-2c6503da6a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:40.153572 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e653d12-d993-44a8-b894-2c6503da6a17 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:40.154280 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e653d12-d993-44a8-b894-2c6503da6a17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:40.156239 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e653d12-d993-44a8-b894-2c6503da6a17 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:40.175846 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1454/2919] 10.222.0.42 () {60 vars in 1143 bytes} [Thu Jul 21 23:18:40 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:18:40.179819 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e084f-443c-476e-b219-c3b793edd423 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:40.190524 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e084f-443c-476e-b219-c3b793edd423 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:40.195653 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ab0cf-9732-45c9-b290-af18a0d4450f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:40.196392 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ab0cf-9732-45c9-b290-af18a0d4450f admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:40.196776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ab0cf-9732-45c9-b290-af18a0d4450f admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:40.197156 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ab0cf-9732-45c9-b290-af18a0d4450f admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:40.197952 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ab0cf-9732-45c9-b290-af18a0d4450f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:40.199587 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ab0cf-9732-45c9-b290-af18a0d4450f admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:40.211617 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1466/2920] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:18:40.215827 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e653d12-d993-44a8-b894-2c6503da6a17 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:40.226080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e653d12-d993-44a8-b894-2c6503da6a17 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:40.230955 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd22ea-4280-4a6f-a6c7-557988f435f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:40.231745 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd22ea-4280-4a6f-a6c7-557988f435f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:40.232030 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd22ea-4280-4a6f-a6c7-557988f435f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:40.232305 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd22ea-4280-4a6f-a6c7-557988f435f9 admin admin] PATH_INFO: `/v3/projects/46f8c8b78d2248f092d48fcc328c3ec0/users/e131188e4e4546dc9c132313eb42f287/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:40.248909 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd22ea-4280-4a6f-a6c7-557988f435f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f8c8b78d2248f092d48fcc328c3ec0, user_id=e131188e4e4546dc9c132313eb42f287, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:40.251595 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd22ea-4280-4a6f-a6c7-557988f435f9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:40.276320 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1455/2921] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:18:40 2022] PUT /identity/v3/projects/46f8c8b78d2248f092d48fcc328c3ec0/users/e131188e4e4546dc9c132313eb42f287/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:18:40.278180 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ab0cf-9732-45c9-b290-af18a0d4450f admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:40.289286 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ab0cf-9732-45c9-b290-af18a0d4450f admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:40.348001 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3e5fb-d685-4295-891b-0baeea670975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:40.348704 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3e5fb-d685-4295-891b-0baeea670975 admin admin] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:40.348997 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3e5fb-d685-4295-891b-0baeea670975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:40.349275 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3e5fb-d685-4295-891b-0baeea670975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:40.350008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3e5fb-d685-4295-891b-0baeea670975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:40.352100 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3e5fb-d685-4295-891b-0baeea670975 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:40.362143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1467/2922] 10.222.0.42 () {58 vars in 1121 bytes} [Thu Jul 21 23:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:18:40.367337 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd22ea-4280-4a6f-a6c7-557988f435f9 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:40.378105 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd22ea-4280-4a6f-a6c7-557988f435f9 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:40.383672 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6a84c-593d-41c4-94ec-d58c55ac6db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:40.384597 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6a84c-593d-41c4-94ec-d58c55ac6db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:40.384951 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6a84c-593d-41c4-94ec-d58c55ac6db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:40.385287 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6a84c-593d-41c4-94ec-d58c55ac6db8 admin admin] PATH_INFO: `/v3/projects/46f8c8b78d2248f092d48fcc328c3ec0/users/e131188e4e4546dc9c132313eb42f287/roles/f957e09d047b47cfa0891d25b4181263` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:40.394434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6a84c-593d-41c4-94ec-d58c55ac6db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f8c8b78d2248f092d48fcc328c3ec0, user_id=e131188e4e4546dc9c132313eb42f287, role_id=f957e09d047b47cfa0891d25b4181263)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:40.397242 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6a84c-593d-41c4-94ec-d58c55ac6db8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:40.408569 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1456/2923] 10.222.0.42 () {60 vars in 1482 bytes} [Thu Jul 21 23:18:40 2022] PUT /identity/v3/projects/46f8c8b78d2248f092d48fcc328c3ec0/users/e131188e4e4546dc9c132313eb42f287/roles/f957e09d047b47cfa0891d25b4181263 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:18:44.763544 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e3a8-98ee-4ccf-8559-cfb582fbd5f3 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:44.764768 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e3a8-98ee-4ccf-8559-cfb582fbd5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:44.765227 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e3a8-98ee-4ccf-8559-cfb582fbd5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:44.810169 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-28a5e3a8-98ee-4ccf-8559-cfb582fbd5f3 None None] MFA Rules not processed for user `e131188e4e4546dc9c132313eb42f287`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:44.811315 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-28a5e3a8-98ee-4ccf-8559-cfb582fbd5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:44.820999 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:44.916530 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1468/2924] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:44.930661 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6a84c-593d-41c4-94ec-d58c55ac6db8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:44.932433 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c8b6a84c-593d-41c4-94ec-d58c55ac6db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:45.005343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6a84c-593d-41c4-94ec-d58c55ac6db8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:45.009763 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05067e1-3490-4d04-bb7b-19a459ae25db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b67db080874f3ebb508239c7a8ccf8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:45.010508 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05067e1-3490-4d04-bb7b-19a459ae25db service glance] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:45.010791 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05067e1-3490-4d04-bb7b-19a459ae25db service glance] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:45.011063 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05067e1-3490-4d04-bb7b-19a459ae25db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:45.038392 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:18:45.071793 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05067e1-3490-4d04-bb7b-19a459ae25db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:45.073948 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05067e1-3490-4d04-bb7b-19a459ae25db service glance] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:45.083162 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1457/2925] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:18:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:18:45.224803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8849be-811b-4550-a793-f0e509912cbe None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:45.224803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8849be-811b-4550-a793-f0e509912cbe None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:45.224803 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8849be-811b-4550-a793-f0e509912cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:45.228361 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-7c8849be-811b-4550-a793-f0e509912cbe None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:45.229634 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-7c8849be-811b-4550-a793-f0e509912cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:45.286864 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1469/2926] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 23:18:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:18:45.334298 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05067e1-3490-4d04-bb7b-19a459ae25db service glance] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:18:45.336630 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c05067e1-3490-4d04-bb7b-19a459ae25db service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:45.406453 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05067e1-3490-4d04-bb7b-19a459ae25db service glance] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:18:45.411355 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2df86-7dfc-4ddf-b607-b329e7084f78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:18:45.411989 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2df86-7dfc-4ddf-b607-b329e7084f78 service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:45.412275 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2df86-7dfc-4ddf-b607-b329e7084f78 service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:45.412591 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2df86-7dfc-4ddf-b607-b329e7084f78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:45.421203 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2df86-7dfc-4ddf-b607-b329e7084f78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:18:45.423015 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2df86-7dfc-4ddf-b607-b329e7084f78 service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:18:45.431606 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1458/2927] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 23:18:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 23:18:54.151150 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f3757-5d0c-4dea-8a78-2b7f052555e0 None None] REQUEST_METHOD: `POST` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:18:54.151713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f3757-5d0c-4dea-8a78-2b7f052555e0 None None] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:18:54.151713 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f3757-5d0c-4dea-8a78-2b7f052555e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:18:54.170788 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.auth.core [None req-359f3757-5d0c-4dea-8a78-2b7f052555e0 None None] MFA Rules not processed for user `4c2cdf754ca24afd88329cd456f1c9d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:18:54.172005 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-359f3757-5d0c-4dea-8a78-2b7f052555e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:18:54.178986 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1470/2928] 10.222.0.42 () {58 vars in 957 bytes} [Thu Jul 21 23:18:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:18:55.428107 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=48 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:19:00.192867 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cc27b-2f49-42fd-abe4-f64c317b70df None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:19:00.193424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cc27b-2f49-42fd-abe4-f64c317b70df None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:19:00.193424 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543cc27b-2f49-42fd-abe4-f64c317b70df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:19:00.211864 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-543cc27b-2f49-42fd-abe4-f64c317b70df None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:19:00.213428 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-543cc27b-2f49-42fd-abe4-f64c317b70df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:19:00.225278 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1459/2929] 10.222.0.42 () {58 vars in 1009 bytes} [Thu Jul 21 23:19:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:19:00.238550 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f3757-5d0c-4dea-8a78-2b7f052555e0 None None] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:19:00.251274 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f3757-5d0c-4dea-8a78-2b7f052555e0 None None] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:19:00.256282 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199eecef-9a8b-4fa6-a07e-25ea35da61c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:19:00.256970 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eecef-9a8b-4fa6-a07e-25ea35da61c7 service swift] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:19:00.257258 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eecef-9a8b-4fa6-a07e-25ea35da61c7 service swift] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:19:00.257537 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eecef-9a8b-4fa6-a07e-25ea35da61c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:19:00.266563 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199eecef-9a8b-4fa6-a07e-25ea35da61c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:19:00.268776 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199eecef-9a8b-4fa6-a07e-25ea35da61c7 service swift] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:19:00.279377 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1471/2930] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 23:19:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 23:19:10.206434 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:19:10.276273 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=61 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:19:26.751538 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cc27b-2f49-42fd-abe4-f64c317b70df None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:19:26.752999 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-543cc27b-2f49-42fd-abe4-f64c317b70df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:19:26.807747 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543cc27b-2f49-42fd-abe4-f64c317b70df None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:19:26.811437 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6cb0f4-16c6-4c2e-a45e-c0e607745a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a120d9e7e204dc5b972fd9402897b41', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:19:26.812088 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cb0f4-16c6-4c2e-a45e-c0e607745a1c service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:19:26.812371 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cb0f4-16c6-4c2e-a45e-c0e607745a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:19:26.812678 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6cb0f4-16c6-4c2e-a45e-c0e607745a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:19:26.822080 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6cb0f4-16c6-4c2e-a45e-c0e607745a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:19:26.823516 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6cb0f4-16c6-4c2e-a45e-c0e607745a1c service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:19:26.832894 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1460/2931] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:19:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:19:36.828690 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=26 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:20:39.245871 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199eecef-9a8b-4fa6-a07e-25ea35da61c7 service swift] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:20:39.247894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-199eecef-9a8b-4fa6-a07e-25ea35da61c7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:20:39.370872 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199eecef-9a8b-4fa6-a07e-25ea35da61c7 service swift] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:20:39.377427 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a9d0d-7268-4e13-8b66-1900c65ab371 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c77f6ccc7cb84c04ac532007a04a0f5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:20:39.378589 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a9d0d-7268-4e13-8b66-1900c65ab371 service nova] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:20:39.378942 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a9d0d-7268-4e13-8b66-1900c65ab371 service nova] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:20:39.379280 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a9d0d-7268-4e13-8b66-1900c65ab371 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:20:39.381417 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-2b2a9d0d-7268-4e13-8b66-1900c65ab371 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:20:39.398556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a9d0d-7268-4e13-8b66-1900c65ab371 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:20:39.400455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a9d0d-7268-4e13-8b66-1900c65ab371 service nova] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:20:39.414716 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1472/2932] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:20:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:20:47.281770 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73585de-109d-4e69-8a63-771b18c5ebc9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:20:47.282197 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73585de-109d-4e69-8a63-771b18c5ebc9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:20:47.282343 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73585de-109d-4e69-8a63-771b18c5ebc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:20:47.337907 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c73585de-109d-4e69-8a63-771b18c5ebc9 None None] MFA Rules not processed for user `d53f6748398d45358f7b2d75ec273dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:20:47.339111 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c73585de-109d-4e69-8a63-771b18c5ebc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:20:47.357215 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1461/2933] 10.222.0.42 () {58 vars in 967 bytes} [Thu Jul 21 23:20:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:20:47.396496 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a9d0d-7268-4e13-8b66-1900c65ab371 service nova] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:20:47.417695 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a9d0d-7268-4e13-8b66-1900c65ab371 service nova] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:20:47.423541 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba079810-59d5-4aff-bdda-962c6fefcfb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fec30ef0b8c41dca643cfefebfe904b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:20:47.424187 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba079810-59d5-4aff-bdda-962c6fefcfb0 service swift] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:20:47.424507 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba079810-59d5-4aff-bdda-962c6fefcfb0 service swift] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:20:47.424786 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba079810-59d5-4aff-bdda-962c6fefcfb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:20:47.434371 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba079810-59d5-4aff-bdda-962c6fefcfb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:20:47.435791 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba079810-59d5-4aff-bdda-962c6fefcfb0 service swift] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:20:47.444147 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1473/2934] 10.222.0.42 () {60 vars in 1384 bytes} [Thu Jul 21 23:20:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 23:20:50.878029 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73585de-109d-4e69-8a63-771b18c5ebc9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:20:50.879448 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c73585de-109d-4e69-8a63-771b18c5ebc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:20:50.961651 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73585de-109d-4e69-8a63-771b18c5ebc9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:20:50.967415 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397ec76-a22c-4ca3-bb00-19ac6065fd33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:20:50.968129 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397ec76-a22c-4ca3-bb00-19ac6065fd33 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:20:50.968407 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397ec76-a22c-4ca3-bb00-19ac6065fd33 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:20:50.968710 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397ec76-a22c-4ca3-bb00-19ac6065fd33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:20:50.970480 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1397ec76-a22c-4ca3-bb00-19ac6065fd33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:20:50.983137 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397ec76-a22c-4ca3-bb00-19ac6065fd33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:20:50.984761 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397ec76-a22c-4ca3-bb00-19ac6065fd33 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:20:50.995117 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1462/2935] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:20:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:20:54.130459 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba079810-59d5-4aff-bdda-962c6fefcfb0 service swift] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:20:54.205957 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba079810-59d5-4aff-bdda-962c6fefcfb0 service swift] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:20:54.210593 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:20:54.211472 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:20:54.211757 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:20:54.212033 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] PATH_INFO: `/v3/users/4c2cdf754ca24afd88329cd456f1c9d2` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:20:54.213252 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c2cdf754ca24afd88329cd456f1c9d2)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:20:54.215354 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:20:54.260748 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c2cdf754ca24afd88329cd456f1c9d2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.269294 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c2cdf754ca24afd88329cd456f1c9d2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.269959 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c2cdf754ca24afd88329cd456f1c9d2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.271204 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:20:54.278894 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c2cdf754ca24afd88329cd456f1c9d2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.279810 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c2cdf754ca24afd88329cd456f1c9d2'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.297929 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1474/2936] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:20:54 2022] DELETE /identity/v3/users/4c2cdf754ca24afd88329cd456f1c9d2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:20:54.473252 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397ec76-a22c-4ca3-bb00-19ac6065fd33 service placement] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:20:54.485023 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397ec76-a22c-4ca3-bb00-19ac6065fd33 service placement] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:20:54.544783 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:20:54.545698 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:20:54.545967 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:20:54.546241 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] PATH_INFO: `/v3/projects/d72850690b7f447fbbb9a30e1bc1fc78` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:20:54.547458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d72850690b7f447fbbb9a30e1bc1fc78)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:20:54.549458 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:20:54.591709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:20:54.595702 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd72850690b7f447fbbb9a30e1bc1fc78', 'request_id': 'req-79dc38a6-ebef-4449-b266-df2b92b2d992'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.601660 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd72850690b7f447fbbb9a30e1bc1fc78', 'request_id': 'req-79dc38a6-ebef-4449-b266-df2b92b2d992'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.609339 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd72850690b7f447fbbb9a30e1bc1fc78', 'request_id': 'req-79dc38a6-ebef-4449-b266-df2b92b2d992'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.620124 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] The token cache is being invalidate because project d72850690b7f447fbbb9a30e1bc1fc78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:20:54.620420 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.621079 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.623269 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1463/2937] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:20:54 2022] DELETE /identity/v3/projects/d72850690b7f447fbbb9a30e1bc1fc78 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:20:54.627455 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:20:54.628878 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:20:54.642215 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e75ba-c9a8-4efc-aba3-76f870b87534 admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:20:54.698570 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:20:54.700087 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:20:54.700616 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:20:54.700908 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] PATH_INFO: `/v3/users/e131188e4e4546dc9c132313eb42f287` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:20:54.702106 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e131188e4e4546dc9c132313eb42f287)` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:20:54.704008 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:20:54.749143 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e131188e4e4546dc9c132313eb42f287'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.756954 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e131188e4e4546dc9c132313eb42f287'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.757556 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e131188e4e4546dc9c132313eb42f287'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.765218 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e131188e4e4546dc9c132313eb42f287'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.765955 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.notifications [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e131188e4e4546dc9c132313eb42f287'} {{(pid=80316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:54.782979 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1475/2938] 10.222.0.42 () {58 vars in 1223 bytes} [Thu Jul 21 23:20:54 2022] DELETE /identity/v3/users/e131188e4e4546dc9c132313eb42f287 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:20:54.954285 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:20:54.955709 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:20:54.970883 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc38a6-ebef-4449-b266-df2b92b2d992 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:20:55.027579 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2841614037940fea030f1d6c1eca499', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5705979642f541aa9229c56505a6ee6e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:20:55.028364 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:20:55.028676 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:20:55.028953 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba admin admin] PATH_INFO: `/v3/projects/46f8c8b78d2248f092d48fcc328c3ec0` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:20:55.030137 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f8c8b78d2248f092d48fcc328c3ec0)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:20:55.032044 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:20:55.079211 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f8c8b78d2248f092d48fcc328c3ec0', 'request_id': 'req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:55.084861 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f8c8b78d2248f092d48fcc328c3ec0', 'request_id': 'req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:55.092412 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f8c8b78d2248f092d48fcc328c3ec0', 'request_id': 'req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:55.104265 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba admin admin] The token cache is being invalidate because project 46f8c8b78d2248f092d48fcc328c3ec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:20:55.104584 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:55.105202 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e6efb0dc-c976-4bef-bc41-2c8f75ef72ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:20:55.107222 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1464/2939] 10.222.0.42 () {58 vars in 1232 bytes} [Thu Jul 21 23:20:54 2022] DELETE /identity/v3/projects/46f8c8b78d2248f092d48fcc328c3ec0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:21:04.779539 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=64,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:21:05.100104 dv-cinder-850696-1-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=59,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:23:31.524611 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] Authenticating user token {{(pid=80316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:23:31.526088 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:23:31.645025 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f738982-f47f-4d06-99cc-445df030e67c admin admin] Validating token access rules against request {{(pid=80316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:23:31.706777 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55271fa4-be8b-4ca8-a136-7f60f5b4f8ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705ff0e73e95499e873b1df2a24fefe8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0638fe2ce7a40299f2d654f58ed535c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:23:31.707346 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55271fa4-be8b-4ca8-a136-7f60f5b4f8ba service placement] REQUEST_METHOD: `GET` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:23:31.707632 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55271fa4-be8b-4ca8-a136-7f60f5b4f8ba service placement] SCRIPT_NAME: `/identity` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:23:31.707951 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55271fa4-be8b-4ca8-a136-7f60f5b4f8ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:23:31.710404 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.fernet_utils [None req-55271fa4-be8b-4ca8-a136-7f60f5b4f8ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:23:31.722523 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55271fa4-be8b-4ca8-a136-7f60f5b4f8ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:23:31.724245 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55271fa4-be8b-4ca8-a136-7f60f5b4f8ba service placement] RBAC: Authorization granted {{(pid=80316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:23:31.734876 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: [pid: 80316|app: 0|req: 1476/2940] 10.222.0.42 () {60 vars in 1365 bytes} [Thu Jul 21 23:23:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:23:41.730242 dv-cinder-850696-1-iscsi devstack@keystone.service[80316]: DEBUG dbcounter [-] [80316] Writing DB stats keystone:SELECT=69 {{(pid=80316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}